News / #long-context Tag Long Context 378 articles archived under #long-context · RSS Sign in to follow arXiv — Machine Learning research 3h ago MARCH: Scaling Recurrent Memory with Content-Routed State Anchors arXiv:2608.12435v1 Announce Type: new Abstract: Transformers owe much of their strong long-context retrieval capability to a token-level memory that grows with context length. This flexibility, however, incurs a quadratic computation complexity during training and a key--value… 15 r/LocalLLaMA community 12h ago You could purchase a Desktop with 2TB of DDR5 - It only sets you back some $200k+ Just watched Wendell's (level1 techs) latest video on the HP Z8 Fury desktop workstation and was curious how you could configure it. And oh boy, there's an option for 2TB which costs some $211k just for the RAM alone. But the real interesting part with the latest price hikes for… 21 arXiv — Machine Learning research 1d ago Disentangling the Expressivity of RoPE arXiv:2608.11909v1 Announce Type: new Abstract: Two accounts recur in explanations of the success of rotary position embeddings (RoPE). Expressivity studies associate periodic position information with modular predicates, whereas mechanistic and long-context studies emphasize… 16 arXiv — NLP / Computation & Language research 1d ago Lost in Compaction: Evaluating Side-Constraint Loss under Context Compaction arXiv:2608.11242v1 Announce Type: new Abstract: When the context window is under pressure, LLM systems compact prior context to continue ongoing tasks. We identify a class of user-issued instructions, Session Constraints (SCs), such as "do not delete any emails until I confirm,"… 27 arXiv — NLP / Computation & Language research 1d ago Information Abundance Paradox: Long-Context Training Undermines Parametric Knowledge arXiv:2608.12218v1 Announce Type: new Abstract: Large language models are increasingly trained and deployed with long contexts that span documents, code repositories, and interaction histories. This scaling reflects the implicit assumption that training on longer contexts will… 32 Hugging Face Daily Papers research 1d ago CoinRAG: Contextualized Information Nugget KV Cache Reuse for Long-Context RAG Abstract CoinRAG improves retrieval-augmented generation efficiency and accuracy by reusing fine-grained semantic nugget caches instead of full chunks. Generated by thinkingmachines/Inkling-Small Recent optimization studies on Retrieval-Augmented Generation (RAG) have exploited… 4 Vercel — AI dev-tools 1d ago GLM 5.2 free for eve agents through August 27 via Blackbox on AI Gateway GLM 5.2 , the open-weights coding model from Z.ai with a 1M-token context window, is free for eve agents through August 27, served by Blackbox AI on AI Gateway . New eve agents come with GLM 5.2 as their default model. Use npx eve@latest init my-agent to get started. Existing… 35 arXiv — NLP / Computation & Language research 2d ago Position Encoding in Transformers: From Absolute and Relative Methods to Rotary Position Embeddings and Long-Context Scaling arXiv:2608.10021v1 Announce Type: new Abstract: Self-attention models content-dependent interactions between tokens but does not by itself encode token order. Position encoding addresses this limitation by introducing absolute coordinates, relative distances, or… 11 arXiv — NLP / Computation & Language research 2d ago Cracks in the Foundation: Seemingly Minor Architectural Choices Impact Long Context Extension arXiv:2608.10296v1 Announce Type: new Abstract: One might imagine that architectural variations within the dense transformer paradigm have a limited effect on accuracy. However, we demonstrate that this is not the case in the long context setting. Specifically, we show that a… 32 Vercel — AI dev-tools 2d ago Grok 4.6 now available on AI Gateway Grok 4.6 from SpaceXAI is now available on AI Gateway . The model has a 500K token context window and accepts text and image inputs. Grok 4.6 supports low, medium, high, and xhigh reasoning levels and defaults to high. To use Grok 4.6, set model to xai/grok-4.6 in the AI SDK :… 29 r/LocalLLaMA community 3d ago I ran Muse Glimmer @ 1M context - All tests passed. Heeeey all! I just completed some fun tests with Muse Glimmer, I thought I'd let you know. In fact, the summary below was written by Muse itself! I ran a 2× DGX Spark cluster and got Meta's day-old Muse Glimmer 30B running the day after release — then pushed its context from the… 36 arXiv — Machine Learning research 3d ago RippleKV: Cross-Layer KV Cache Allocation via Perturbation Propagation arXiv:2608.08684v1 Announce Type: new Abstract: Long-context LLM inference is bottlenecked by KV cache memory, yet distributing a limited cache budget across layers remains challenging. Existing methods rely on proxies such as layer depth, attention statistics, or representation… 34 arXiv — Machine Learning research 3d ago DistillCache: KL-Guided Adaptive KV-Cache Eviction for Memory-Efficient LLM Inference arXiv:2608.08878v1 Announce Type: new Abstract: Transformer-based large language models (LLMs) achieve strong performance across many tasks, but their Key-Value (KV) cache grows linearly with sequence length, creating a severe memory bottleneck for long-context inference.… 32 Hugging Face Daily Papers research 3d ago Motif 3: Technical Report Abstract Motif 3 is a large sparse mixture-of-experts language model using grouped differential latent attention and specialized training techniques to achieve strong reasoning, coding, and long-context performance. Generated by thinkingmachines/Inkling-Small We introduce Motif… 16 r/LocalLLaMA community 3d ago Please Share Your Experience About Muse Glimmer I have a classic test for local LLM's. I asked for 8 ball pool game with only one HTML file and Muse Glimmer spend 21k Token(I m using full context so 128k) and only created a 220 lines of HTML and said its done. With my experience its not even close to Qwen 3.6 27B and we are… 19 NVIDIA Developer Blog official-blog 3d ago Run Local Agentic AI Workflows with Meta’s Muse Glimmer on NVIDIA Meta returns to the open source ecosystem with the release of Muse Glimmer, a 30B open-weight dense model with a 120K+ context window built for local AI... 37 r/LocalLLaMA community 3d ago 1M context with 17 GB model in 24 GB VRAM: "for the first time I was able to load a context of almost 1M tokens and extract 7 needles from various parts of the text" https://preview.redd.it/xxjh11f38jih1.png?width=1852&format=png&auto=webp&s=76850ed51e29a8bc86c2ca718d4320075eed4363 Just wanted to share a user report that I found to be very interesting. Some person with an intriguing name manu69x managed to run 1M context on a single RTX 3090… 30 arXiv — Machine Learning research 4d ago Every Cache Entry Earns Its Place: Global Allocation of Resolution and Coverage for KV Cache Compression arXiv:2608.07001v1 Announce Type: new Abstract: As large language models (LLMs) process increasingly long contexts, KV cache storage and repeated access have become a major bottleneck. Existing KV cache compression methods rely on predefined, fixed compression rules and are… 30 arXiv — NLP / Computation & Language research 4d ago Autonomy-of-Heads: Data-Free Sparse Attention from Frozen Query-Key Geometry arXiv:2608.06849v1 Announce Type: new Abstract: Long-context LLM inference is bottlenecked by quadratic attention computation and growing KV-cache costs. Existing sparse attention and KV-compression methods typically decide which tokens or heads to preserve from runtime… 27 arXiv — NLP / Computation & Language research 4d ago CoinRAG: Contextualized Information Nugget KV Cache Reuse for Long-Context RAG arXiv:2608.07458v1 Announce Type: new Abstract: Recent optimization studies on Retrieval-Augmented Generation (RAG) have exploited chunk-level KV cache reuse to avoid processing long retrieved contexts for higher efficiency, while significant information redundancy and noise… 35 r/LocalLLaMA community 5d ago 128GB vs 256gb of ram Imagine you have 128gb of VRAM. what accompanying ram capacity you would choose (DDR4 8channel)? For example Deepseek v4 flash in q8 takes around 170GB + 12GB Dflash + ~10GB per 1m context so it’s under 200gb. so 128 + 128 should be good But for something like MiMo… 5 r/LocalLLaMA community 5d ago Tesla V100 Qwen3.6 27B Performance Looking for V100 users to share your config and it's performance. GPU: Tesla V100 PCIE 32Gb Qwen3.6 27B Q4_K_M + Q8_0 MTP 128K context length Pi coding agent llama.cpp model preset: [*] spec-default = 1 ctx-size = 131072 mmap = 1 kv-unified = 1 n-gpu-layers = 999 threads = 18… 14 arXiv — NLP / Computation & Language research 7d ago QEvict: Recoverable Quantized KV Eviction for Attention-Drift-Robust Long-Context Decoding arXiv:2608.05326v1 Announce Type: cross Abstract: Autoregressive large language model inference is increasingly constrained by the memory footprint of the Key-Value (KV) cache. A dominant line of work reduces this footprint by evicting tokens that appear unimportant under… 28 arXiv — Machine Learning research 7d ago Is Self-Pretraining really useful to improve diagnosis in medical Time Series? arXiv:2608.06122v1 Announce Type: new Abstract: Inspired by recent evidence that transformer architectures benefit from Self-PreTraining (SPT) on long-context benchmarks, we investigate whether similar gains extend to multimodal, multivariate, and even simple univariate medical… 23 r/LocalLLaMA community 7d ago Final optimization: from ~10 tok/s to ~15 tok/s on DeepSeek-V4-Flash-0731 at 128K ctx - 1 RTX 3090 J'ai consacré beaucoup de temps à l'optimisation de DeepSeek-V4-Flash-0731 GGUF sur une seule RTX 3090. Mon exigence absolue pour chaque configuration était la suivante : Le modèle doit rester utilisable avec une fenêtre de contexte de 128 000 jetons. J'ai testé les différentes… 4 arXiv — Machine Learning research 8d ago Spend Bits Where Queries Look: KV Cache Vector Quantization with Attention-Preserving Transforms arXiv:2608.04074v1 Announce Type: new Abstract: Long-context LLM decoding reads the key-value (KV) cache at every step. Loading it takes longer than computing attention over it, so throughput is bandwidth-bound. Hence, reducing the cache size can raise both decoding speed and… 13 arXiv — NLP / Computation & Language research 8d ago Training-Free Hashing-Based Attention via Binary Principal Components arXiv:2608.04405v1 Announce Type: cross Abstract: Long-context large language models (LLMs) are increasingly deployed in real-world applications, yet self-attention remains a major efficiency bottleneck -- especially during decoding -- due to the necessity of repeatedly… 24 arXiv — NLP / Computation & Language research 8d ago Relevant but Incomplete: Referential Dangling as a Paradigm-Level Failure Mode in Hard Prompt Compression arXiv:2608.04569v1 Announce Type: new Abstract: Hard prompt compression reduces long-context inference cost by independently scoring tokens, sentences, or chunks and retaining the highest-scoring units under a budget. We identify a structural failure in this procedure:… 17 arXiv — NLP / Computation & Language research 8d ago Chained Recursive Language Models for Multi-Iteration Reasoning arXiv:2608.05124v1 Announce Type: new Abstract: Long context reasoning in large language models (LLMs) is usually constrained by the fact that a single inference trajectory has to simultaneously explore the context, store intermediate state, verify evidence, and produce the… 20 Vercel — AI dev-tools 8d ago Ling 3.0 Tiny is now available on AI Gateway Ling 3.0 Tiny from ANT Group is now on AI Gateway, free to use till 8:00am PT on 8/14. Ling 3.0 Tiny takes the free slot from Ling 3.0 Flash . Ling 3.0 Tiny is a MOE model with 7.9B total parameters and about 1.3B active per token, a 256K token context window, and up to 32K… 6 r/LocalLLaMA community 8d ago LFM2.5-2.6B on a OnePlus 13 at 17 tok/s ~ Pure CPU As you all know the model is 2.69B parameters with a 128K context window and purpose-built for multi-step agent workflows. What you are seeing is the Q4_K_M GGUF running on my own inference engine built from scratch. The TUI is my own device probe suite running through ADB… 27 Hugging Face Daily Papers research 8d ago Are the Financial Reasoning from LLMs Credible? A Real World Test over Long-Horizon Statements Abstract Do Large Language Models (LLMs) possess genuine structural reasoning, or merely rely on surface-level pattern matching? The financial domain, demanding numerical precision and multi-step logic over long contexts, is an ideal testbed. Existing benchmarks fail to capture… 10 arXiv — Machine Learning research 9d ago Output-Aware Rotation for INT2 KV-Cache Quantization arXiv:2608.02691v1 Announce Type: new Abstract: The key-value (KV) cache has become a major memory and bandwidth bottleneck in long-context large language model inference, making ultra-low-bit quantization increasingly important. However, existing rotation-based INT2 methods… 23 arXiv — NLP / Computation & Language research 9d ago AnchorKV: Anchor-Residual KV Cache Compression arXiv:2608.02901v1 Announce Type: cross Abstract: The key-value (KV) cache is the primary memory bottleneck in long-context LLM inference. Existing approaches attack it from opposite ends: eviction methods permanently discard tokens, degrading performance whenever a discarded… 37 arXiv — Machine Learning research 9d ago SAKI: Score-Aware Low-Rank Key Indexing for Long-Context KV Retrieval arXiv:2608.03228v1 Announce Type: new Abstract: Existing low rank KV cache methods preserve either model weights or key variance, neither of which directly reflects the attention scores used during inference. We derive the expected attention score distortion caused by rank r key… 11 arXiv — Machine Learning research 9d ago TimeRLM: Recursive Language Models Enable Precise Anomaly Localization in Long-Context Time-Series arXiv:2608.03391v1 Announce Type: new Abstract: Precise anomaly localization over long-context time series is a crucial task in monitoring applications across clinical care, industrial operations, financial services, and logistics, where brief evidence may hide inside long spans… 31 arXiv — NLP / Computation & Language research 9d ago PI-Mem: Pushing Long-Context Reasoning to 3.6M Tokens with Parallel-Iterative Memory arXiv:2608.03048v1 Announce Type: new Abstract: Long-context reasoning remains a critical bottleneck for large language models, as recent recurrent-memory approaches face two inherent challenges: sequential chunk-wise updates can overwrite early critical evidence with later… 15 r/LocalLLaMA community 9d ago A 2.6B model with tool calling and 128K context now runs at 30 tok/s on a phone Liquid AI released LFM2.5-2.6B today, and this might be more relevant to local AI than another massive model most people cannot run. The model is only 2.69B parameters, has 128K context, supports tool calling and was post-trained specifically for multi-step agent workflows. The… 23 llama.cpp releases dev-tools 9d ago b10273 sampler : remove "full-context windows" from history-based samplers ( #26524 ) Resolve -1 to 1024 instead of ctx-len for samplers Because of backend-sampling we initialize samplers before the complete llama_context is there. Therefore, we cannot infer the resolved context length… 36 r/LocalLLaMA community 9d ago [Deepseek-V4-Flash-0731] Full 1M context on a single RTX5090 + DDR5 Desktop Setup with VLLM CPU/Ram Offloading, ~800 tps pp & 15+ tps decode [Agentic Coding] First of all, obviously I took some help from AI to type this post and this is the topic that enabled me to accomplish all that: https://old.reddit.com/r/LocalLLaMA/comments/1veow4b/deepseek_v4flash_284b_moe_at_33_toks_single_68/ This post of mine is based on the link above. My… 11 arXiv — NLP / Computation & Language research 10d ago AgentMemBench: A Systematic Benchmark for Evaluating Long-Term Memory Management Strategies in Conversational AI Agents arXiv:2608.00009v1 Announce Type: new Abstract: Long-term memory remains a critical bottleneck for conversational AI agents, whose finite context windows cannot support coherent recall across thousands of turns. We present AgentMemBench, a unified, reproducible benchmark… 32 arXiv — NLP / Computation & Language research 10d ago SeDeM: Selective Decompression of Hidden-State Memories for Long-Context Question Answering arXiv:2608.00311v1 Announce Type: new Abstract: Long-context inference with large language models (LLMs) is costly: self-attention during prefill scales quadratically with sequence length, and the key-value (KV) cache grows with the number of processed tokens. Larger context… 15 arXiv — NLP / Computation & Language research 10d ago S$^4$R: Selective Sampling, Subspaces, and Sparse Reconstruction for Compressed Long-Context KV Caching arXiv:2608.00528v1 Announce Type: new Abstract: The growth of context window lengths in Large Language Models (LLMs) significantly enhances their long-context capabilities but incurs prohibitive memory costs due to the Key-Value (KV) cache. Although low-rank compression of KV… 7 arXiv — NLP / Computation & Language research 10d ago LongChart VQA: A Comprehensive Benchmark for MLLMs with Complex Multi-Chart Reasoning arXiv:2608.01328v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) are rapidly evolving with expanded context windows and stronger reasoning capabilities, enabling multi-chart understanding and multi-step inference. These abilities are increasingly… 10 arXiv — NLP / Computation & Language research 10d ago Learning What to Remember: Test-Time Training via Context Distillation arXiv:2608.01672v1 Announce Type: new Abstract: Effective long-context modeling is not merely about retaining more of the past, but about preserving the information that may prove relevant later. Test-time training (TTT) is an appealing approach that performs online parameter… 28 arXiv — NLP / Computation & Language research 10d ago Understanding Sparse Attention Selectivity in Long-Context Foundation Models via Counterfactual Evaluation arXiv:2608.01676v1 Announce Type: new Abstract: Sparse attention is widely deployed in long-context serving stacks, yet no framework audits how discarding blocks changes the influence of specific content on model output. We first establish that the phenomenon is real and causal:… 18 arXiv — NLP / Computation & Language research 11d ago ThinkReset: Learnable Intermediate Interface Construction for Bounded-Context Long-Horizon Reasoning arXiv:2607.28642v1 Announce Type: cross Abstract: Long chain-of-thought reasoning improves performance on complex problems, but it also introduces redundancy accumulation, context overflow, and error anchoring. We argue that under bounded context windows, the core bottleneck is… 31 arXiv — NLP / Computation & Language research 11d ago Are the Financial Reasoning from LLMs Credible? A Real World Test over Long-Horizon Statements arXiv:2607.28661v1 Announce Type: new Abstract: Do Large Language Models (LLMs) possess genuine structural reasoning, or merely rely on surface-level pattern matching? The financial domain, demanding numerical precision and multi-step logic over long contexts, is an ideal… 30 arXiv — NLP / Computation & Language research 11d ago ResKV: Reconstructing Omitted Attention Contributions for Fixed-Budget KV Cache Compression arXiv:2607.29591v1 Announce Type: new Abstract: KV cache compression is essential for efficient long-context inference. Existing eviction methods permanently discard unselected tokens and consequently remove their aggregate contribution to attention. Merging-based alternatives… 10 Vercel — AI dev-tools 12d ago Qwen 3.8 Max now available on Vercel AI Gateway Qwen 3.8 Max is now available on AI Gateway. Qwen 3.8 Max handles text-only and vision-language work in one model, with 2.4 trillion parameters and a context window of up to 1 million tokens. The model is suited for software engineering and office productivity, along with visual… 13 Page 1 of 8 · 378 articles Older →
