Conventional LLMs keep the full KV cache loaded during decoding, causing a severe GPU memory bottleneck for ultra-long context serving. In this report, we propose Lookahead Sparse Attention (LSA), a novel inference paradigm powered by a Neural Memory Indexer built upon the DeepSeek-V4 architecture. Rather than passively attending to all historical tokens, LSA proactively predicts future context demands and preserves only the query-critical KV chunks in the GPU memory. Crucially, we instantiate this architecture via a backbone-free decoupled training strategy. By formulating the indexer as a standard dual-encoder architecture, we train it independently using standard retrieval training frameworks without ever loading the massive backbone model into GPU memory.<br>We demonstrate that this \"less is more\" paradigm significantly maximizes serving efficiency while acting as an effective attention denoiser in tasks that rely on long-term global memory. Across primary long-context evaluation suites (e.g., LongBench-v2, LongMemEval, and RULER), FM-DS-V4 compresses the average physical KV cache footprint down to merely 13.5% of the full-context baseline, while consistently preserving or slightly elevating downstream accuracy (+0.6% absolute margin on average). Crucially, at extreme 500K scales, FlashMemory suppresses the physical KV cache overhead by over 90% without destabilizing the backbone's core reasoning capacities.<br><a href=\"https://cdn-uploads.huggingface.co/production/uploads/64674375946476c5d215382d/tKnnxVKfu14aellSKdt4i.png\" rel=\"nofollow\"><img src=\"https://cdn-uploads.huggingface.co/production/uploads/64674375946476c5d215382d/tKnnxVKfu14aellSKdt4i.png\" alt=\"experiment_charts\"></a></p>\n<p><a href=\"https://cdn-uploads.huggingface.co/production/uploads/64674375946476c5d215382d/dmVaJPS2B8mhaJNHiTrn4.png\" rel=\"nofollow\"><img src=\"https://cdn-uploads.huggingface.co/production/uploads/64674375946476c5d215382d/dmVaJPS2B8mhaJNHiTrn4.png\" alt=\"幻灯片1\"></a></p>\n","updatedAt":"2026-06-09T03:58:36.425Z","author":{"_id":"64674375946476c5d215382d","avatarUrl":"/avatars/0f0b98080c064a17d003da8a22a047cc.svg","fullname":"Yan Wang","name":"libertywing","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":2,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8134397864341736},"editors":["libertywing"],"editorAvatarUrls":["/avatars/0f0b98080c064a17d003da8a22a047cc.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2606.09079","authors":[{"_id":"6a278f466dde1c5ef75bd034","name":"Yan Wang","hidden":false},{"_id":"6a278f466dde1c5ef75bd035","name":"Qifan Zhang","hidden":false},{"_id":"6a278f466dde1c5ef75bd036","name":"Jiachen Yu","hidden":false},{"_id":"6a278f466dde1c5ef75bd037","name":"Tian Liang","hidden":false},{"_id":"6a278f466dde1c5ef75bd038","name":"Dongyang Ma","hidden":false},{"_id":"6a278f466dde1c5ef75bd039","name":"Xiang Hu","hidden":false},{"_id":"6a278f466dde1c5ef75bd03a","name":"Zibo Lin","hidden":false},{"_id":"6a278f466dde1c5ef75bd03b","name":"Chunyang Li","hidden":false},{"_id":"6a278f466dde1c5ef75bd03c","name":"Zhichao Wang","hidden":false},{"_id":"6a278f466dde1c5ef75bd03d","name":"Jia Li","hidden":false},{"_id":"6a278f466dde1c5ef75bd03e","name":"Yujiu Yang","hidden":false},{"_id":"6a278f466dde1c5ef75bd03f","name":"Haitao Mi","hidden":false},{"_id":"6a278f466dde1c5ef75bd040","name":"Dong Yu","hidden":false}],"publishedAt":"2026-06-08T00:00:00.000Z","submittedOnDailyAt":"2026-06-09T00:00:00.000Z","title":"FlashMemory-DeepSeek-V4: Lightning Index Ultra-Long Context via Lookahead Sparse Attention","submittedOnDailyBy":{"_id":"64674375946476c5d215382d","avatarUrl":"/avatars/0f0b98080c064a17d003da8a22a047cc.svg","isPro":false,"fullname":"Yan Wang","user":"libertywing","type":"user","name":"libertywing"},"summary":"Conventional LLMs keep the full KV cache loaded during decoding, causing a severe GPU memory bottleneck for ultra-long context serving. In this report, we propose Lookahead Sparse Attention (LSA), a novel inference paradigm powered by a Neural Memory Indexer built upon the DeepSeek-V4 architecture. Rather than passively attending to all historical tokens, LSA proactively predicts future context demands and preserves only the query-critical KV chunks in the GPU memory. Crucially, we instantiate this architecture via a backbone-free decoupled training strategy. By formulating the indexer as a standard dual-encoder architecture, we train it independently using standard retrieval training frameworks without ever loading the massive backbone model into GPU memory.\n We demonstrate that this \"less is more\" paradigm significantly maximizes serving efficiency while acting as an effective attention denoiser in tasks that rely on long-term global memory. Across primary long-context evaluation suites (e.g., LongBench-v2, LongMemEval, and RULER), FM-DS-V4 compresses the average physical KV cache footprint down to merely 13.5% of the full-context baseline, while consistently preserving or slightly elevating downstream accuracy (+0.6% absolute margin on average). Crucially, at extreme 500K scales, FlashMemory suppresses the physical KV cache overhead by over 90% without destabilizing the backbone's core reasoning capacities.","upvotes":18,"discussionId":"6a278f466dde1c5ef75bd041","githubRepo":"https://github.com/libertywing/FlashMemory-Deepseek-V4","githubRepoAddedBy":"user","ai_summary":"Lookahead Sparse Attention with Neural Memory Indexer reduces GPU memory usage for long-context LLM inference while maintaining accuracy through proactive KV cache management and decoupled training.","ai_keywords":["LLMs","KV cache","decoding","Neural Memory Indexer","Lookahead Sparse Attention","dual-encoder architecture","attention denoiser","long-context evaluation","FlashMemory"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct","githubStars":1,"organization":{"_id":"66543b6e420092799d2f625c","name":"tencent","fullname":"Tencent","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/5dd96eb166059660ed1ee413/Lp3m-XLpjQGwBItlvn69q.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"64674375946476c5d215382d","avatarUrl":"/avatars/0f0b98080c064a17d003da8a22a047cc.svg","isPro":false,"fullname":"Yan Wang","user":"libertywing","type":"user"},{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"},{"_id":"6645e248b03e00c6a0f61b88","avatarUrl":"/avatars/97af419819044ff90e1937916d6377b1.svg","isPro":false,"fullname":"zhichaowang","user":"zacks917","type":"user"},{"_id":"6645ecc2047aafc187d68cba","avatarUrl":"/avatars/0580d23367cdb18327494115a581494e.svg","isPro":false,"fullname":"zacks","user":"Jadeislaw0917","type":"user"},{"_id":"663a3f88ec1aafe3d62f6201","avatarUrl":"/avatars/3081e50ffa0b97d6c1d01f149ef28d40.svg","isPro":false,"fullname":"zhang qi fan","user":"Bklight999","type":"user"},{"_id":"643645baaa4211ef553f613c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/643645baaa4211ef553f613c/svUVeTqwLf5ZurprdTOUC.jpeg","isPro":false,"fullname":"TimLeung","user":"skytliang","type":"user"},{"_id":"6683b1082ca1c52c2756c458","avatarUrl":"/avatars/04c060438a54dcc658b7584b93369461.svg","isPro":false,"fullname":"Zhang","user":"WestZhang","type":"user"},{"_id":"682b8ee1ae6a4f496ac16c66","avatarUrl":"/avatars/6bfe8240e12f70ffd6e1e7f6214fe7ea.svg","isPro":false,"fullname":"zhaoyijia","user":"JarvisZyj93","type":"user"},{"_id":"66b858351893d14ee829b054","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/66b858351893d14ee829b054/b2ASTdQP3vvq2epNO96Fm.jpeg","isPro":false,"fullname":"xyliu","user":"xiaoyuanliu","type":"user"},{"_id":"676e13d5940dd17d669ddb5e","avatarUrl":"/avatars/aa3c3eaffad47a4ece774580536f4e5a.svg","isPro":false,"fullname":"llm_gen","user":"llm-gen","type":"user"},{"_id":"641129818573c51c0458b793","avatarUrl":"/avatars/d4bc67c160a07146cf41c614678aa36b.svg","isPro":false,"fullname":"Tianqing Fang","user":"tqfang229","type":"user"},{"_id":"66e3d2a41c79fc2e516f7611","avatarUrl":"/avatars/74e9454fbc14dc1bfc888a72bce70747.svg","isPro":false,"fullname":"Daisy","user":"Daisyq","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"66543b6e420092799d2f625c","name":"tencent","fullname":"Tencent","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/5dd96eb166059660ed1ee413/Lp3m-XLpjQGwBItlvn69q.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2606/2606.09079.md"}">
FlashMemory-DeepSeek-V4: Lightning Index Ultra-Long Context via Lookahead Sparse Attention
Authors: ,
,
,
,
,
,
,
,
,
,
,
,
Abstract
Lookahead Sparse Attention with Neural Memory Indexer reduces GPU memory usage for long-context LLM inference while maintaining accuracy through proactive KV cache management and decoupled training.
Conventional LLMs keep the full KV cache loaded during decoding, causing a severe GPU memory bottleneck for ultra-long context serving. In this report, we propose Lookahead Sparse Attention (LSA), a novel inference paradigm powered by a Neural Memory Indexer built upon the DeepSeek-V4 architecture. Rather than passively attending to all historical tokens, LSA proactively predicts future context demands and preserves only the query-critical KV chunks in the GPU memory. Crucially, we instantiate this architecture via a backbone-free decoupled training strategy. By formulating the indexer as a standard dual-encoder architecture, we train it independently using standard retrieval training frameworks without ever loading the massive backbone model into GPU memory.
We demonstrate that this "less is more" paradigm significantly maximizes serving efficiency while acting as an effective attention denoiser in tasks that rely on long-term global memory. Across primary long-context evaluation suites (e.g., LongBench-v2, LongMemEval, and RULER), FM-DS-V4 compresses the average physical KV cache footprint down to merely 13.5% of the full-context baseline, while consistently preserving or slightly elevating downstream accuracy (+0.6% absolute margin on average). Crucially, at extreme 500K scales, FlashMemory suppresses the physical KV cache overhead by over 90% without destabilizing the backbone's core reasoning capacities.
Community
Conventional LLMs keep the full KV cache loaded during decoding, causing a severe GPU memory bottleneck for ultra-long context serving. In this report, we propose Lookahead Sparse Attention (LSA), a novel inference paradigm powered by a Neural Memory Indexer built upon the DeepSeek-V4 architecture. Rather than passively attending to all historical tokens, LSA proactively predicts future context demands and preserves only the query-critical KV chunks in the GPU memory. Crucially, we instantiate this architecture via a backbone-free decoupled training strategy. By formulating the indexer as a standard dual-encoder architecture, we train it independently using standard retrieval training frameworks without ever loading the massive backbone model into GPU memory.
We demonstrate that this "less is more" paradigm significantly maximizes serving efficiency while acting as an effective attention denoiser in tasks that rely on long-term global memory. Across primary long-context evaluation suites (e.g., LongBench-v2, LongMemEval, and RULER), FM-DS-V4 compresses the average physical KV cache footprint down to merely 13.5% of the full-context baseline, while consistently preserving or slightly elevating downstream accuracy (+0.6% absolute margin on average). Crucially, at extreme 500K scales, FlashMemory suppresses the physical KV cache overhead by over 90% without destabilizing the backbone's core reasoning capacities.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2606.09079 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2606.09079 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.