I ran Muse Glimmer @ 1M context - All tests passed.

Heeeey all! I just completed some fun tests with Muse Glimmer, I thought I'd let you know. In fact, the summary below was written by Muse itself!

I ran a 2× DGX Spark cluster and got Meta's day-old Muse Glimmer 30B running the day after release — then pushed its context from the trained 131K all the way to 1M with YaRN, verifying retrieval at every rung. Sharing config + results since the "131,072+" hint in the model card turned out to be very real.

Setup

• Hardware: 2× NVIDIA DGX Spark (GB10, 128 GB unified each, ~273 GB/s), ConnectX-7 direct link between them
• Engine: llama.cpp master (day-1 muse_glimmer support), built from source with CUDA sm_121 + GGML_RPC
• Model: official Muse-Glimmer-30B-GGUF K-Quant-Dynamic (~18.3 GiB) + official mmproj (vision) + official DFlash drafter
• Spec decode: --spec-type draft-dflash --spec-draft-n-max 15 (block-diffusion drafter)
• Context extension: --rope-scaling yarn --rope-scale <2/4/8> --yarn-orig-ctx 131072 plus --override-kv muse-glimmer.context_length=int:<N> (llama.cpp caps at trained length otherwise)
• Yes, we also ran it split across both Sparks with llama.cpp RPC — no reason beyond liking to cluster things for fun. Our daily driver on this hardware is DeepSeek-V4-Flash-0731 on official vLLM, TP=2 over RDMA at full 1M context, which is the fair comparison point.

Results

Needle-in-haystack (3 needles at 10/50/90% depth):

Speed:

• Single Spark: ~10.5 tok/s baseline decode → 36–38 tok/s with DFlash (~3×, matching Meta's claimed 3.1× on a 5090); prefill ~700 tok/s short-context, ~390 tok/s deep into an 832K prompt; ×4 concurrent ≈ 57 tok/s aggregate per node
• RPC split across both Sparks: 25–28 tok/s decode — ~30% slower than single-node. A 20 GB model doesn't need two nodes, and layer-split pays a network hop per token. Fun, not fast.

Other:

• Coding: 7/7 on our small execution-checked suite (LRU cache, RFC4180 CSV parser, rotated binary search, etc.), both nodes
• Vision works via the official mmproj (shapes/colors/text reading)
• Weights + drafter + vision + full 1M KV ≈ ~60 GB on one Spark

Why the YaRN stretch works so well on this model (our theory)

Muse's config is unusual: RoPE exists only on the 39 sliding-window layers (2,048-token window), while the 13 global full-attention layers have no positional encoding at all (NoPE). So when you YaRN-stretch 8×:

• the local layers barely notice — within a 2K window, relative positions are identical at any document length
• the long-range layers, the ones actually bridging 800K tokens, never had rotation embeddings to break in the first place

Result: retrieval held perfect at every rung we tested, where conventional full-RoPE architectures typically fray. The tiny KV (2 KV heads, mostly sliding layers) is what makes 1M practical on this class of hardware at all.

Conclusion

• Muse Glimmer 30B is a genuinely strong local agentic model, and its usable context goes far beyond the spec sheet: verified 3/3 retrieval at 832K tokens with nothing more exotic than YaRN flags and a metadata override
• DFlash spec decode is the difference between "unusable dense-30B speeds" and "pleasant" on bandwidth-limited hardware — ~3× for free
• Cluster-splitting via llama.cpp RPC works but is slower than single-node — run one instance per box instead
• Eagerly awaiting vLLM support for muse_glimmer so we can run it TP=2 over NCCL/RDMA like our DeepSeek setup — the moment it lands we'll A/B and report back