focus-llama: a llama.cpp fork implementing Declarative Attention (arXiv:2609.02737)

I forked llama.cpp to implement Declarative Attention (arXiv:2609.02737, Google DeepMind and KAIST AI). The model declares in its own output which context chunks it needs <focus magic\_chunks="N">), and the engine listens and restricts what the following tokens can attend to. No scorer, no training: just prompting plus an engine that reacts to the tags. According to their paper, the overall decode time for the response can be reduced to 0.71× (Gemma) and 0.77× (Qwen) compared to vanilla. Those are the paper's vLLM numbers; I haven't benchmarked the fork yet (no accuracy benchmark either).

= What works =

- llama-server can drop KV token ranges for a request da_rm / da_rm_at), either mid-prefill or after prefill.
- Tag-driven mode: given a chunk layout from the client, the server parses the model's first <focus magic\_chunks="N"> tag during generation and drops the other chunks at that point (one-shot, one-way).
- A second-sequence mode da_b, needs --kv-unified) leaves the original sequence intact.
- Checked with first-token logprobs on small smoke tests (a Qwen hybrid/GDN model and a tiny dense one).

= Why a fork =

- Stock llama-server has no way to touch KV ranges mid-generation, so this needed server changes.
- llama.cpp has no paged block table (the paper's vLLM version rewrites it), so masking alone doesn't reduce the KV that gets read.
- Real skipping needs kernel support or compaction. From reading the source, single-token decode on CUDA doesn't skip masked chunks today.
- On hybrid models only the attention layers are restricted (same as the paper); the recurrent state is untouched.

Repo: https://github.com/edwardyoon/focus-llama

Feedback welcome, especially from anyone who has worked on llama.cpp KV/recurrent memory.