News / #long-context Tag Long Context 378 articles archived under #long-context · RSS Sign in to follow Hugging Face Daily Papers research 29d ago SynthDocBench: Controlled Benchmark for Long-Context Visual Document Understanding Abstract Vision language models (VLMs) have achieved strong performance on visual document understanding benchmarks such as DocVQA, ChartQA, and MMLongBench-Doc. However, real-world documents combine multiple factors such as length, layout complexity, modality, and question… 23 arXiv — Machine Learning research 1mo ago Semidirect Fourier Delta Attention: Phase-Controlled Delta Memory with Constructive Chunk-WY Kernels arXiv:2607.11897v1 Announce Type: new Abstract: Linear attention replaces softmax attention's growing KV cache with a fixed recurrent state, but this compression limits exact state tracking and long-context memory. We introduce \emph{Semidirect Fourier Delta Attention} (SFDA), a… 38 arXiv — Machine Learning research 1mo ago LiteTopK: Exploiting the Curse of Dimensionality for a Fused Indexer-TopK Kernel in Long-Context Sparse Attention arXiv:2607.11976v1 Announce Type: new Abstract: Indexer-TopK, the operation to compute the scores and select the top-k candidates, is widely used by sparse attention kernels in large language models and vector retrieval in recommendation systems and vector databases. However,… 34 arXiv — NLP / Computation & Language research 1mo ago A JoLT for the KV Cache: Near-Lossless KV Cache Compression via Joint Tucker and JL-Residual Allocation for LLMs arXiv:2607.12550v1 Announce Type: cross Abstract: The key-value (KV) cache has become the dominant memory cost of transformer inference. It grows with batch size, context length, and depth, and at long context it, rather than the model weights, sets the ceiling on throughput.… 31 arXiv — Machine Learning research 1mo ago What Context Does a Coding Agent Actually Need to Act? arXiv:2607.09691v1 Announce Type: new Abstract: A modern coding agent can hold an entire repository in its context window. Most of its reading is wasted -- and the interesting question is not how much context an agent can use, but what it actually \emph{needs}. We study that… 38 arXiv — NLP / Computation & Language research 1mo ago GEIS: A Generation-Evaluation-Improvement Loop of Agent Skills for Long-Form Article Generation arXiv:2607.11503v1 Announce Type: new Abstract: Long-form article generation remains difficult for large language models because it combines long context, long instructions, and long outputs. Existing multi-agent pipelines such as STORM improve information coverage by simulating… 19 Hugging Face Daily Papers research 1mo ago Self-Guided Test-Time Training for Long-Context LLMs Abstract Long-context processing has become increasingly important for large language models (LLMs), but simply extending the context window does not guarantee effective utilization of long inputs. As input length grows, accuracy often degrades, indicating that models still… 4 arXiv — NLP / Computation & Language research 1mo ago WILDTRACE: Benchmarking Natural Evidence Trails in Long-Context Reasoning arXiv:2607.09328v1 Announce Type: new Abstract: Answering complex questions over long documents frequently requires integrating evidence that the source itself disperses naturally across distant passages. In an incident report, the operating condition, design flaw, and missed… 16 arXiv — NLP / Computation & Language research 1mo ago Self-Guided Test-Time Training for Long-Context LLMs arXiv:2607.09415v1 Announce Type: new Abstract: Long-context processing has become increasingly important for large language models (LLMs), but simply extending the context window does not guarantee effective utilization of long inputs. As input length grows, accuracy often… 31 arXiv — NLP / Computation & Language research 1mo ago Evaluating Retrieval-Augmented Generation vs. Long-Context Input for Clinical Reasoning over EHRs arXiv:2508.14817v2 Announce Type: replace Abstract: Objective: To evaluate whether retrieval-augmented generation (RAG) can serve as an efficient alternative to long-context prompting for clinical reasoning over electronic health records (EHRs). Methods: We defined three… 27 arXiv — NLP / Computation & Language research 1mo ago REAL: REtrieval-reAsoning and Logic-constructed Attention Behaviors for Long-Context KV Cache Compression arXiv:2508.15806v2 Announce Type: replace Abstract: The growing sequence length of large language models poses significant challenges for key-value (KV) caches. Existing state-of-the-art cache eviction methods primarily analyze the inference behavior of attention heads in… 4 r/LocalLLaMA community 1mo ago Running Qwen3.5-122B on Mac Studio 96GB: Fixed 3 bugs that made long-context inference usable Hey everyone, I recently switched from DS4 Flash to Qwen3.5-122B on my M3 Ultra Mac Studio for long-context agentic coding. While the model fit better, I hit a wall where follow-up messages took 3-5 minutes to start generating (cold fills) despite having a "warm" context. Turns… 22 r/LocalLLaMA community 1mo ago CTX: How far can you reasonably go with Qwen 3.6 27B? How far can i stretch the context window with Qwen 3.6 27B (using Q8_0) before it gets too unreliable? I am at 100k right now and i am not quite statisfied. Other than not quantizing KV cache, is there anything else that can be done to make the model more stable over longer CTX?… 18 r/LocalLLaMA community 1mo ago tencent/HiLS-Attention-7B · Hugging Face HiLS-Attention is a chunk-wise sparse attention mechanism that learns chunk selection end-to-end under the language-modeling loss, enabling native sparse training for efficient long-context modeling. This repository hosts the 7B checkpoint continued-trained on top of an… 7 Hugging Face Daily Papers research 1mo ago Jet-Long: Efficient Long-Context Extension with Dynamic Bifocal RoPE Abstract A novel zero-shot method called Jet-Long enables efficient long-context processing for large language models by dynamically adapting rescaling factors and utilizing a bifocal attention mechanism that maintains high performance across varying sequence lengths. Generated… 36 r/LocalLLaMA community 1mo ago [Paper] Sparse Delta Memory: Scaling the State of Linear RNNs through Sparsity Linear attention models allow a fixed state size and a fixed amount of compute per token. However, due to their limited state size, linear attention models fall behind in long-context recall compared to softmax-attention-based transformer architectures. Increasing the state size… 9 arXiv — NLP / Computation & Language research 1mo ago Uncertainty-gated selection for block-sparse attention arXiv:2607.07724v1 Announce Type: cross Abstract: Block-sparse attention scales long-context language models by replacing the O(N^2) softmax with a per-query top-k selection over key blocks. This cutoff is myopic: when the k-th and (k+1)-th blocks are nearly tied in score, the… 7 arXiv — Machine Learning research 1mo ago Jet-Long: Efficient Long-Context Extension with Dynamic Bifocal RoPE arXiv:2607.07740v1 Announce Type: new Abstract: Modern LLMs are increasingly deployed in long-context applications such as retrieval-augmented generation, repository-level coding, and agentic workflows whose accumulated reasoning and tool traces routinely push the input an order… 37 arXiv — Machine Learning research 1mo ago Linear Attention Architectures: Mechanisms, Trade-offs, and Cross-Layer Routing arXiv:2607.07953v1 Announce Type: new Abstract: Self-attention lets each token retrieve information from the full context, but its quadratic cost in sequence length limits training and inference at long context. This paper presents a comparative study of softmax attention and… 10 r/LocalLLaMA community 1mo ago Locally run assistant on a w-10 board on a local Xiaozhi server Hello, Locallama! "long-time" member here, from the days when the max context window doubled from 2K to a whooping 4K! Now I feel like I am living in a dream world, doing things I could not have imagined back then. This is Belochka, or Bella, running on a Mac and projected to an… 31 r/LocalLLaMA community 1mo ago Stripping terminal noise from agent context via a lazy-loaded local CLI layer. Looking for brutal feedback on this heuristic. When building long-running coding agents, terminal output is one of the fastest ways to poison a context window. If an agent runs an intensive build, an install command, or a massive search ( grep / find ), it easily generates hundreds of lines of raw log noise. The agent reads… 7 Hugging Face Daily Papers research 1mo ago Sparse Delta Memory: Scaling the State of Linear RNNs through Sparsity Abstract Sparse Delta Memory extends gated linear RNNs with sparse addressing to dramatically increase hidden state capacity for improved long-context learning and retrieval while maintaining computational efficiency. Generated by Qwen/Qwen2.5-Coder-32B-Instruct Linear attention… 30 arXiv — Machine Learning research 1mo ago Fractal KV-Cache Archives: Lossless Symbolic Storage with In-Place Retrieval for Long-Context LLM Inference arXiv:2607.07144v1 Announce Type: new Abstract: The key-value (KV) cache dominates the memory cost of long-context autoregressive inference, and a growing body of work compresses it through quantization, eviction, or offloading. We study a complementary question: once a… 36 arXiv — Machine Learning research 1mo ago Sparse Delta Memory: Scaling the State of Linear RNNs through Sparsity arXiv:2607.07386v1 Announce Type: new Abstract: Linear attention models allow a fixed state size and a fixed amount of compute per token. However, due to their limited state size, linear attention models fall behind in long-context recall compared to softmax-attention-based… 5 arXiv — NLP / Computation & Language research 1mo ago Nectar: Neural Estimation of Cached-Token Attention via Regression arXiv:2605.09778v2 Announce Type: replace-cross Abstract: Evaluating softmax attention over a fixed long context requires reading every cached key-value pair for each new query token. For a given context (a book, a manual, a legal corpus) the attention output is a deterministic… 18 Vercel — AI dev-tools 1mo ago Muse Spark 1.1 is now available on AI Gateway Muse Spark 1.1 from Meta is now available on AI Gateway . It is a multimodal reasoning model with a 1M token context window built for agentic tasks, accepting text, image, video, PDF, and audio inputs. Muse Spark 1.1 plans and orchestrates work across tools and services,… 34 arXiv — Machine Learning research 1mo ago PatchOptic for Shared-State LLM Workflows with Projected Views and Verified Structured Updates arXiv:2607.05483v1 Announce Type: new Abstract: Agentic workflows often operate over shared, structured state. Because LLM context windows are limited, each model invocation is typically shown only the state fragment needed for the current workflow step, a pattern commonly known… 7 arXiv — NLP / Computation & Language research 1mo ago Benchmarking KV-Cache Optimizations across Task Quality and System Performance for Long-Context Serving arXiv:2607.05399v1 Announce Type: new Abstract: Large language model serving is increasingly limited by KV-cache growth under long-context workloads, yet existing KV-cache compression techniques are difficult to compare because they were evaluated on different models, tasks,… 30 arXiv — NLP / Computation & Language research 1mo ago ResonatorLM: Causal Resonant Field Mixing for Efficient Long-Context Language Modelin arXiv:2607.05583v1 Announce Type: new Abstract: Contemporary language models are dominated by the transformer architecture, which leverages self-attention mechanisms to enable more efficient, parallelized training across a wide set of documents and corpora. This has allowed… 20 arXiv — NLP / Computation & Language research 1mo ago Inject or Navigate? Token-Efficient Retrieval for LLM Analysis of Transactional Legal Documents arXiv:2607.05764v1 Announce Type: new Abstract: Answering questions over a set of transactional legal documents is most simply done by injecting the whole corpus into the LLM's context window on every query. That baseline maximises retrieval recall, but its token footprint… 31 arXiv — NLP / Computation & Language research 1mo ago LongCrafter: Towards Diverse Long-Context Understanding via Evidence-Graph-Guided Instruction Synthesis arXiv:2607.06160v1 Announce Type: new Abstract: Synthesizing long-context supervised fine-tuning (SFT) data is a scalable way to enhance the long-context understanding of large language models (LLMs), yet existing approaches share three limitations: narrow task coverage,… 10 Hugging Face Daily Papers research 1mo ago Hierarchical Sparse Attention Done Right: Toward Infinite Context Modeling Abstract Hierarchical Landmark Sparse Attention enables efficient long-context language modeling by learning chunk selection end-to-end, achieving performance comparable to full attention while extrapolating beyond training context lengths. Generated by… 35 Hugging Face Daily Papers research 1mo ago SeKV: Resolution-Adaptive KV Cache with Hierarchical Semantic Memory for Long-Context LLM Inference Abstract SeKV introduces a resolution-adaptive semantic KV cache that compresses context into entropy-guided spans stored across GPU-CPU memory hierarchies, enabling efficient long-context processing with minimal memory overhead and preserved token-level detail. Generated by… 34 arXiv — Machine Learning research 1mo ago Training Hybrid Block Diffusion Language Models with Partial Bidirectionality arXiv:2607.02805v1 Announce Type: new Abstract: High-throughput long-context generation is one of the central challenges for large language models. Generation is typically memory-bandwidth-bound rather than compute-bound: each decoding step must stream the accumulated key/value… 32 arXiv — NLP / Computation & Language research 1mo ago Hierarchical Sparse Attention Done Right: Toward Infinite Context Modeling arXiv:2607.02980v1 Announce Type: new Abstract: Scaling modern large language models (LLMs) to long contexts is limited by the quadratic computation cost, and poor length extrapolation of dense attention. Chunk-wise sparse attention offers a promising alternative, but all… 36 arXiv — NLP / Computation & Language research 1mo ago SelfMem: Self-Optimizing Memory for AI Agents arXiv:2607.03726v1 Announce Type: new Abstract: While current AI agents support increasingly long context windows, tool use, and skill execution for long-horizon tasks, they still require memory systems to effectively leverage historical experience. Existing memory frameworks… 10 r/LocalLLaMA community 1mo ago Got my Ascent GX10 two days ago, ran REAP-pruned NVFP4 DeepSeek-V4-Flash on a single Spark, and it stays consistent at long context Got my Ascent GX10 two days ago and spent the last couple of days pushing a REAP-pruned NVFP4 DeepSeek-V4-Flash setup on a single Spark by patching the eugr/spark-vllm-docker image. Credit where it’s due: the REAPs were done by 0xSero . I’m just the person who wired it up,… 36 r/LocalLLaMA community 1mo ago Llama-Server is Throwing Away Your Perfectly Good KV Caches, and How to Fix It Introduction I watched 2.49 GB of state restore from disk in 1.23 seconds — and then get thrown away. llama-server's slot save/restore promises exactly what long-context work on budget hardware needs: park a session on disk, bring it back later without paying the prefill tax… 24 r/MachineLearning community 1mo ago Why I built a proactive context curator instead of a compactor — and what I got wrong for three months [P] Two ways to handle a context window that's filling up. Reactive: wait until it's full, then compact everything. Proactive: be picky about what gets added every turn so noise never piles up in the first place. Most coding agents take the reactive path. I spent months building the… 19 r/LocalLLaMA community 1mo ago I benchmarked 13 models at 65K-128K context to find out what actually matters for agentic workloads I benchmarked 13 models at 65K-128K context to find out what actually matters for agentic workloads — prefill dominates everything, and KV head count beats parameter count I've been running local LLMs for agentic workflows (tool use, coding agents, RAG) and kept seeing people… 36 r/LocalLLaMA community 1mo ago Appreciation post! Brought my first 3090 and been running Qwen 27B with 200K context, couldn't be happier. I'm using the club 3090 configuration and I highly recommend it! https://github.com/noonghunna/club-3090/tree/master Thanks to the community!   submitted by   /u/Top_Outlandishness78… 16 r/LocalLLaMA community 1mo ago We run 14 local-first agent harnesses with all working memory in TOON instead of Markdown — measured benchmarks (including the number that doesn't favor us) and the upstream bug report we got wrong We moved our agent fleet's working memory off Markdown and onto TOON (Token-Oriented Object Notation) in December 2025 and just wrote up what 14 harnesses taught us. The honest numbers (tiktoken o200k, 100 uniform CRM records): - TOON 2,068 tokens vs row-object JSON 3,074… 28 Hacker News — AI on Front Page community 1mo ago Google Books (or similar) all book scans – $200k bounty (2025) Article URL: https://software.annas-archive.gl/AnnaArchivist/annas-archive/-/work_items/234 Comments URL: https://news.ycombinator.com/item?id=48786838 Points: 201 # Comments: 95 12 r/LocalLLaMA community 1mo ago Comparing local inference speeds across a few real setups people are running (3090 vs 5090 vs dual 6000) Pulled together token rates from a few different local rigs people have reported running lately, just to get a sense of what's realistic at each hardware tier(source discord group) Qwen3.6 27B on a single 3090 (Q4/Q8 MTP, 128k ctx): ~50 tok/s inference, ~950 tok/s prompt… 29 r/MachineLearning community 1mo ago Proposal: Use semantic compression as input diffusion to read sessions larger than the context window [R] I've been trying to come up with a solution for keeping extremely long ai sessions coherent. Sometimes there is too much substance to risk compaction. With so much buzz around diffusion going on it got me thinking, what if we treat the context like a progressive render,… 7 r/LocalLLaMA community 1mo ago A fully local, self-hosted repo index for coding agents (Rust, MIT, runs offline) If you run local models for coding, feeding them repo context without blowing the context window is half the battle. I built basemind to index a repo locally and serve it over MCP: a code map across 300+ languages, git history and blame, and document RAG over 90+ formats, all on… 15 r/LocalLLaMA community 1mo ago Follow-up: DeepSeek V4 Flash on 2x RTX PRO 6000 finishes real coding tasks faster than Sonnet and Opus, at about Sonnet quality This is a follow-up to post about which local models stay fast deep into long context and I learned a lot from people here. I kept measuring after that and it turned into a proper indie coding bench. With DeepSeek V4 Flash running on vLLM it lands around Sonnet quality and it… 24 r/LocalLLaMA community 1mo ago Follow-up: GLM-5.2 NVFP4 on four DGX Sparks — the MTP mystery is solved, and it's now ~24 tok/s at 128K context Follow-up: GLM-5.2 NVFP4 on four DGX Sparks — the MTP mystery is solved, and it's now ~24 tok/s at 128K context This is a follow-up to my earlier post about running GLM-5.2 NVFP4 on 4x DGX Spark at 128K context. Short version of that post: 128K worked at ~15 tok/s with MTP1, and… 27 arXiv — NLP / Computation & Language research 1mo ago Multi-Head Recurrent Memory Agents arXiv:2607.01523v1 Announce Type: cross Abstract: Recurrent memory agents extend LLMs to arbitrarily long contexts by iteratively consolidating input into a fixed-size memory window. Despite their scalability, these agents exhibit a well-documented reliability problem:… 34 Hugging Face Daily Papers research 1mo ago Morphing into Hybrid Attention Models Abstract FlashMorph is an efficient layer selection method that formulates hybrid layer selection as a budget-constrained optimization problem, using morphable models and linearization regularization to improve long-context efficiency in Transformers. Generated by… 5 Page 3 of 8 · 378 articles ← Newer Older →
