The bear can dance: Qwen 3.8 27B on one 3090 for 3 weeks

TL;DR: Local agent loop, ~21 days, one RTX 3090. Task was pretty much "build a CUDA inference engine for optimized for yourself on this GPU arch." Got working kernels and benches, not a win over llama.cpp. ~12 human messages. Compaction ate ~83 hours.

Old joke: you don’t criticize how well the bear dances, you’re surprised it dances at all.

Setup: Qwen 3.8 27B Q4, Q8 KV, 200k context, deepseek harness, written rulebook: roles, handoffs, when to ping me, don't copy llama.cpp, don't declare the task impossible alone. I don't write CUDA. Nudges were basically "llama.cpp does ~700 prefill on this card, you're at ~250, try harder."

Run: Unsupervised for days at a stretch, then escalate when the rules say so. Near day 6 it had several kernels and prefill stuck around 250 tps; same pattern later. Stops were mostly protocol, not the model wandering off. A protocol that's more empowering can probably keep this going indefinitely.

Suicide loop: Same 3090 has to host the agents (vLLM) and run the engine under test. Both want the full GPU. Kill vLLM wrong and every agent goes dark, leave it up during a bench and you OOM. The rulebook requires a fixed handoff script: stop vLLM, bench, start vLLM, poll health until it's back, write STATE. One subworker treated that as optional, kept killing vLLM outside the window, crashed the orchestrator, then did it again. A worker shutting down the brain that runs it. Harness also hard-crashed once; I restarted that by hand. Fixable with locks and "only this role may touch vllm.sh" protocol-level refinements.

Local tax: 180 subagents, ~230M tokens in+out, ~1.7B cache-read. 699 compactions, ~83 h inside them (~17% of calendar time). Typical compact ~7 min on a ~160k+ token prompt.

Prefill landed ~half of llama.cpp on the same card. Still: weeks of coherent goal-following on a consumer box, it left working kernels, benches, notes, and a long git history. For a local (quantized!) 27B to hold a real engineering goal for that long, I’ll take it. Not a graceful ballerina, but damn this bear can dance!

Dump + rules (~15 GB):
https://huggingface.co/datasets/skeole/qwen-cpp-agent-0-protocol

Backend:
https://github.com/syv-ai/HyperQwen (amazing work by u/iamMess)