The journey of letting Qwen 3.6/3.8 autonomously coding a c compiler.

Hi,

Back in late march I begun playing around with Qwen 3.6 27b and found like everyone else that it's notoriously good at tool calls, where every model I tried before just derailed after a few turns it kept going and felt quite reliable outside of typical behaviors of smaller modells.

I decided to take a crack at it and see if a custom harness that attempted to detect and recitify the modell if it detected repetitions, empty answers etc (some of these issues was later addressed to some extent with updated jinja templates though), but also combining some novel and other just basic ideas that is common among frontier agent harnesses.

After a few weeks I had something that appeared to work quite well for simple test applications/utilities and most issues I encountered was context related- ie. I use llama.cpp as inference engine without context-shift so the harness/orchestrators context management is extremely important and I had a few ideas I wanted to test here.

Around the start of July it was in a shape that I wanted to see how far I could push both the harness and the model itself, the whole idea was to attempt force the model research then execute instead of relying on it's burned in information. as well as apply a very strict general ruleset with multiple subagents, a planner, coder, debugger, researcher, validators etc all focused on their own task to structure, replan and execute.

I gave it a prompt "I want to make a c99 c compiler capable of producing working x64 elfs" and so it started, the first few days (my inference rig is a Tesla P100 + RTX 4070 for the Qwen model, inference speed is around 13-14 tok/s and prefill around 250 token/s ) and then I run Gemma4 12b on an Intel Arc B580 on another machine which is used as the validator most of the time.

Outside some minor tweaks to the orchestration (particularly to the context management and more especially the compaction/pruning, which was very finicky as you easially ended up in a situation where the system spent 6-7 minutes doing prefill and then predicting until the next tool call which once again triggered forced pruning and ruining the KV cache like this repetively, this clearly affected the systems speed negatively- and is by far the main reason it taken 6 weeks, I did not figure a good way to solve this until 3 weeks in), the other major issue I had was when it started with the x86 code generation, this was extremely frustrating as it rather hallucinated opcodes etc than look them up, this was eventually improved by making the coder/debugger system prompts much more rigid and encouraging use of libcapstone etc, but there is no denying that this was the most troublesome area and where it spent the majority of the time for weeks.

Speaking of which the longest period it been running completely uninterrupted has been 1 week, other than that there been lots of improvements and tweaks to both prompts and architecture of the orchestrator (I might describe the orchestrator architecture in another post some day, but I don't want to get into details until/if I open source it, currently it's extremely tailored to my own equipment and I really don't have the energy since I already spent hours almost every day for around 6 months on this to try generalize it)

Anyway long story short- my point with this is just to give an example of that Qwen 3.6/3.8 (I upgraded the model the day 3.8 was released) 27b is extremely capable if steered and given the right circumstances.

(I could be wrong but I believe this may be the most advanced project I've seen it produce so far).

The produced project can be found here: https://github.com/Na1w/tc