Muse-Glimmer 30B Hits ~280 t/s in Real Production Coding
Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.
These numbers were captured during a real feature implementation task in Next.js and Nest.js (adding a theme switching system across components). The structural predictability of UI/state refactoring is likely why DFlash hit such a high draft acceptance rate (~97%).
Here is a quick log analysis and performance summary running Muse-Glimmer-30B (UD-
Q6_K_XL) paired with DFlash (Speculative Decoding) via llama.cpp (llama-server + single RTX 5090).
-ngl 99 -c 200000 --host 0.0.0.0 --port 8080 --timeout 600 --cache-reuse 256 --parallel 1 --flash-attn on --spec-type draft-dflash --spec-draft-n-max 16 --spec-draft-p-min 0.7 --spec-draft-ngl 99 --cache-type-k q8_0 --cache-type-v q8_0 --no-webui --load-mode none --cache-ram 12192 --temperature 0.8 --top-k 30 --top-p 0.95 --min-p 0.05 --repeat-penalty 1.1 --repeat-last-n 64 --reasoning on --chat-template-kwargs {"enable_thinking":true}
Compared to Qwen 3.6 27B: No Chinese language-mixing bugs, no overthinking loops, and concise responses. Its lighter memory footprint at Q6 also freed up more VRAM/RAM for a much larger context size.
| Metric | Measured Value | Notes |
|---|---|---|
| Generation Speed (Peak) | 100 – 287 tokens/sec | Average ~173 t/s across all tasks; ideal for IDE completion |
| Prompt Processing (Short) | 1,000 – 2,100 t/s | ~100–1,000 tokens evaluated in 0.1s – 0.5s |
| Prompt Processing (Long) | 2,654 tokens/sec | 14.3k tokens evaluated in ~5.35 seconds |
| Draft Acceptance Rate | 63.3% – 97.5% | Average 82.3% across 22 test tasks |
| Mean Accepted Draft Length | Up to 14.0 tokens | Multi-token acceptance driving massive acceleration |
| KV Cache Reuse (LCP) | 99.9% (f_sim = 0.999) | Near-instant response on repeated code contexts |
* update: Muse glimmer 30B's seq length is 131,072
[link] [comments]
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.