News / #long-context Tag Long Context 378 articles archived under #long-context · RSS Sign in to follow r/LocalLLaMA community 12d ago LongCat-Flash-Lite-Sparse Is Now Available for Download The weights have now been added to the repo an hour ago. This model is built upon LongCat-Flash-Lite , the differences are that LongCat-Flash-Lite-Sparse : Replaces dense MLA with LongCat Sparse Attention (LSA) Natively supports context lengths of up to 1M tokens (vs 256k for… 13 r/LocalLLaMA community 13d ago What speeds are everyone getting with deepseek v4 flash 0731? What speeds are everyone getting with deepseek v4 flash 0731? I’m getting~200 tps prompt processing / ~11 tps token gen, on 4x5060ti16gb with ddr4 3200 ram at 4-channel, via llamacpp, with context window of 128000, -ub/-b at 4096, “q8” unsloth’s lossless quant   submitted by… 36 NVIDIA Developer Blog official-blog 13d ago Co-Designing AI Model Attention for Fast, Interactive Long-Context Inference As agentic and long-context workloads become common, the context lengths increase and attention consumes a larger share of inference time (Figure 1). Because... 20 llama.cpp releases dev-tools 13d ago b10201 ggml-webgpu: improve flash_attn_vec for quantized KV at long contexts ( #25956 ) improve fa of quantized kv cache Fix some bugs and some comments. fix v type check and some comments Fix build error caused by rebasing editorconfig checking pass Website: https://llama.app… 13 arXiv — Machine Learning research 14d ago Beyond KV Reconstruction: Functional Reconstruction for MLA Draft Models in Speculative Decoding arXiv:2607.27269v1 Announce Type: new Abstract: Multi-head latent attention (MLA) is increasingly important for long-context LLM inference because compact latent states replace the growing key-value (KV) cache and reduce decoding memory traffic. Yet most capable open checkpoints… 16 arXiv — NLP / Computation & Language research 14d ago Recall Before You Rank: Similarity-Guided Top-$K$ Reuse for Efficient Long-Context Attention arXiv:2607.27692v1 Announce Type: new Abstract: Top-$K$ sparse attention reduces the cost of Softmax and value aggregation by attending to only a small subset of key--value (KV) entries. However, identifying this subset still requires scoring the current query against the full… 13 arXiv — NLP / Computation & Language research 14d ago PCAP-LM: An LLM-Native Text Representation for TLS Bulk Traffic Analysis arXiv:2607.28100v1 Announce Type: cross Abstract: Large language models (LLMs) offer powerful reasoning capabilities for network traffic analysis, but standard capture formats and their textual equivalents are prohibitively verbose, overflowing LLM context windows by two orders… 37 r/LocalLLaMA community 14d ago Inkling-Small by thinkingmachines 276B total parameters, 12B active, 1M context window. Blog post: https://thinkingmachines.ai/news/inkling-small/ NVFP4: https://huggingface.co/thinkingmachines/Inkling-Small-NVFP4 GGUF's by Unsloth: https://huggingface.co/unsloth/Inkling-Small-GGUF   submitted by  … 20 arXiv — NLP / Computation & Language research 15d ago Mergeable Model-Side Aggregation States for Long-Context Language Models arXiv:2607.26448v1 Announce Type: new Abstract: A known limitation of long-context language models is their increasingly unreliable performance in non-additive, set-based aggregation as context length grows. Examples include cardinality estimation, set relationships, and grouped… 22 arXiv — NLP / Computation & Language research 15d ago DenseOn with the LateOn: Fully Open Dense and Late-Interaction Models for Multilingual, Long-Context, and Code Search arXiv:2607.27178v1 Announce Type: new Abstract: State-of-the-art retrieval models increasingly rely on closed training data, creating a reproducibility gap. We present an open end-to-end recipe for training retrieval models and study how English supervision transfers to… 21 arXiv — NLP / Computation & Language research 15d ago MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent arXiv:2507.02259v2 Announce Type: replace Abstract: Despite improvements by length extrapolation, efficient attention and memory modules, handling infinitely long documents with linear complexity without performance degradation during extrapolation remains the ultimate challenge… 22 Vercel — AI dev-tools 15d ago AI Gateway: GPT-5.6 pricing and speed updates On AI Gateway , GPT-5.6 Luna and GPT-5.6 Terra are now cheaper and GPT-5.6 Sol is faster. AI Gateway adds no markup on token pricing, so these changes reach you at the upstream rate. The changes apply to both short and long context pricing. Model Change Input: Short context (per… 4 arXiv — NLP / Computation & Language research 16d ago CoSA: Accelerating Long-Context Inference via Proxy-Kernel Co-Designed Sparse Attention arXiv:2607.25291v1 Announce Type: new Abstract: The quadratic cost of self-attention makes long-context inference prohibitively expensive, and proxy-based block-sparse attention has become a practical remedy. Existing methods typically rely on a proxy to predict a binary sparse… 32 arXiv — NLP / Computation & Language research 16d ago GLIDE: Guided Layerwise Hybrid Attention for Efficient LLM Inference arXiv:2607.24788v1 Announce Type: cross Abstract: As Large Language Models scale to increasingly long contexts, the memory I/O and computational overhead of the Key-Value (KV) cache during decoding emerges as the primary throughput bottleneck. To address this, we propose GLIDE,… 36 arXiv — NLP / Computation & Language research 16d ago Addressable Recall Compaction for Long Context-Window Control in AI Agents arXiv:2607.25066v1 Announce Type: cross Abstract: Long-horizon LLM agents accumulate reasoning traces, actions, and tool observations that can eventually exceed a model's fixed context window. Existing compaction methods address this limitation by discarding, summarizing, or… 30 arXiv — NLP / Computation & Language research 16d ago HANDBOOK.md: A Benchmark for Long-Context Agentic Instruction Following arXiv:2607.25398v1 Announce Type: cross Abstract: Language-model agents are increasingly deployed under standing instructions: a system prompt, a policy file, or a skills document is placed in context, and the agent is trusted to let it govern every action that follows. Existing… 6 Hugging Face official-blog 16d ago LFM2.5-Encoders for Fast Long-Context Inference on CPU Back to Articles a]:hidden"> LFM2.5-Encoders for Fast Long-Context Inference on CPU Team Article Published July 28, 2026 Upvote 14 Fernando Fernandes Neto fernandofernandes LiquidAI Edoardo Mosca EdoardoMosca LiquidAI Maxime Labonne mlabonne LiquidAI Leonie Monigatti iamleonie… 18 arXiv — Machine Learning research 17d ago Variational-Ising-Attention (VIA):TailoredAttentionMattersfor Science arXiv:2607.23634v1 Announce Type: new Abstract: Attention enables context modeling via query-key scoring with softmax normalization. Driven by industrial long-context demands, mainstream research has converged toward sparsity and efficiency--yet softmax's independence assumption… 36 arXiv — NLP / Computation & Language research 17d ago INS-ActBench: A Comprehensive Benchmark for Assessing Professional Actuarial Capability of Large Language Models arXiv:2607.24273v1 Announce Type: new Abstract: Large Language Models (LLMs) have shown strong potential in financial reasoning, but existing benchmarks often evaluate domain knowledge, numerical reasoning, long-context understanding, and tool use in separate settings. This… 32 arXiv — NLP / Computation & Language research 17d ago Kimi K3: Open Frontier Intelligence arXiv:2607.24653v1 Announce Type: new Abstract: We introduce Kimi K3, a 2.8T parameter Mixture-of-Experts model with 104 billion activated parameters, native vision capabilities, and a 1-million-token context window. Kimi K3 is built on Kimi Delta Attention and Attention… 20 Hugging Face Daily Papers research 17d ago Kimi K3: Open Frontier Intelligence Abstract We introduce Kimi K3, a 2.8T parameter Mixture-of-Experts model with 104 billion activated parameters, native vision capabilities, and a 1-million-token context window. Kimi K3 is built on Kimi Delta Attention and Attention Residuals, which improve information flow… 7 r/LocalLLaMA community 17d ago First evidence of a pending qwen3.7 open weights release. Qwen3.7-flash is on open router. They referred to Qwen3.6-35b-a3b as Qwen3.6 flash so this is likely a small MoE. The prices are substantially cheaper than 3.6 flash with a native 1M context window.   submitted by   /u/fulgencio_batista [link]   [comments] 37 arXiv — NLP / Computation & Language research 18d ago Learning What Matters: Supervising Sparse Attention Routing with Causal Evidence Sets arXiv:2607.21692v1 Announce Type: cross Abstract: Sparse attention reduces the cost of long contexts by allowing each query to read only selected parts of the input. These selectors are often trained by distilling the attention patterns of a dense teacher, assuming that… 31 arXiv — Machine Learning research 18d ago RIS-Kernel: A Model-Agnostic Architecture for Long-Context LLM Inference via Sparse Attention arXiv:2607.21927v1 Announce Type: new Abstract: Full self-attention in large language models scales as O(N^2), which limits long-context document analysis to 65,536 tokens and requires costly GPU clusters. The Reduced Interaction Sampling (RIS) inference engine addresses this… 31 r/LocalLLaMA community 19d ago Mobile Offline LLMs: What do you use them for? I've spent the last year or so playing around with open source MLX and GGUF models on iPhone hardware. Given the limitations in memory, GPU/CPU/ANE, and in turn the context window I've been trying to figure out the best use cases for them. I've also done a lot of testing with… 13 r/LocalLLaMA community 19d ago Kimi Linear 48B A3B? Just noticed this exists, 1M context MOE with 48B par seems just like what Ive been looking for - it runs pretty damn fast too compared to Qwen 3.6 35B. after some testing it seems capable of producing *not terrible* results but it always tries to go for the minimun possible… 24 r/LocalLLaMA community 19d ago My GX10 died Everything ran fine, I was using UD 3.6 Q6 for 35 and 27B, each 4 concurrent requests at 200K context. I had Dify and Mastra to play around with, Unsloth studio to get around to and vLLM ready for whenever I decided to do some more testing. LLama-swap above lama.cpp and liteLLM… 30 r/LocalLLaMA community 20d ago DKV: Open-source KV-cache compression framework for local LLM inference (CLI + technical report) Hi everyone! Over the past five months I've been working on DKV (DifferentialKV), an open-source project exploring KV-cache compression for long-context local LLM inference. The goal is to reduce KV-cache memory requirements through anchor-based representations, joint low-rank… 21 arXiv — Machine Learning research 21d ago Codec-Gauge: Learning Compression-Friendly Gauges for Transformer KV Caches arXiv:2607.20538v1 Announce Type: new Abstract: Long-context Transformer inference increasingly relies on KV-cache compression or quantization. Prior rotation and transform-coding results suggest that the channel basis of each key/value vector affects how faithfully a fixed… 14 arXiv — NLP / Computation & Language research 21d ago news-crawler-LM: A Small Long-Context Model For High-Quality News Crawling arXiv:2607.21284v1 Announce Type: new Abstract: Extracting structured content from news pages remains challenging due to heterogeneous HTML layouts, inconsistent markup, and substantial boilerplate such as navigation elements and advertisements. Rule-based news crawlers can… 13 Vercel — AI dev-tools 22d ago Ling 3.0 Flash is now available on AI Gateway Ling 3.0 Flash from Ant Group is now available on AI Gateway. The model is free to use for the next three weeks, through August 3rd. Ling 3.0 Flash is a Mixture-of-Experts model with 124B total parameters and about 5.1B active per token. It has a 256K token context window and… 12 r/LocalLLaMA community 22d ago Tokenizer Expansion: Upgrading a Model's Tokenizer in Place - LFM2.5-8B-A1B Today, we're sharing the recipe behind the new tokenizer in LFM2.5-8B-A1B . It upgrades a pre-trained model's tokenizer in place , without retraining from scratch. We doubled the vocabulary from 65K to 128K to fix the languages our original tokenizer split too finely. Blog:… 34 arXiv — NLP / Computation & Language research 23d ago Copy Less, Ground More: Overcoming Repetitive Copying in Long-Context Reasoning via Evidence-Aware Reinforcement Learning arXiv:2607.19345v1 Announce Type: new Abstract: Large language models that generate step-by-step reasoning traces have achieved strong performance on complex tasks, and extending them to long-context settings has emerged as an important frontier. However, we identify a critical… 16 arXiv — Machine Learning research 24d ago High-accuracy Low-Bit KV-Cache Quantization via Local Distribution Restoration arXiv:2607.16248v1 Announce Type: new Abstract: Long-context large language model inference relies on the KV cache to avoid redundant attention computation, but incurs high memory and bandwidth overheads. Low-bit KV-cache quantization reduces this cost, yet it severely degrade… 17 arXiv — Machine Learning research 24d ago More Than Memory: Task-Conditioned Signed FFN Writes in Long-Context Retrieval arXiv:2607.16254v1 Announce Type: new Abstract: FFNs are often treated as parametric memories. In long-context retrieval, however, the sharper question is not only what they store, but whether their native residual writes push the current retrieval state toward or away from the… 10 arXiv — NLP / Computation & Language research 24d ago C$^2$KV: Compressed and Composable KV Cache Reuse for Efficient LLM Inference arXiv:2607.17715v1 Announce Type: new Abstract: Long-context inference is central to modern large language model (LLM) applications such as retrieval-augmented generation and multi-document reasoning. To mitigate the growing inference cost, recent work has explored key-value… 20 arXiv — NLP / Computation & Language research 24d ago SWE-Pruner Pro: The Coder LLM Already Knows What to Prune arXiv:2607.18213v1 Announce Type: new Abstract: Pruning long context for coding agents has been a vital technology for efficient context management. While existing context pruning methods such as SWE-Pruner realize this by attaching a separate code classifier, we find the agent… 8 arXiv — NLP / Computation & Language research 24d ago Is Progressive Disclosure All You Need for Long-Context Agents? arXiv:2607.17598v1 Announce Type: cross Abstract: Long-document question answering usually forces a choice between loading the whole document into the context window and bolting on a separate retriever. Agentic AI suggests a broader option, giving the agent the document path and… 30 Hugging Face Daily Papers research 24d ago SWE-Pruner Pro: The Coder LLM Already Knows What to Prune Abstract Pruning long context for coding agents has been a vital technology for efficient context management. While existing context pruning methods such as SWE-Pruner realize this by attaching a separate code classifier, we find the agent itself encodes internal representations… 22 Vercel — AI dev-tools 24d ago Laguna S 2.1 is now available on AI Gateway Laguna S 2.1 from Poolside is now available on AI Gateway. There are 2 versions of the model available: Free version (256K context window): poolside/laguna-s-2.1-free Paid version (1M context window): poolside/laguna-s-2.1 Laguna S 2.1 is an open-weight Mixture-of-Experts model… 17 arXiv — NLP / Computation & Language research 25d ago VarRate: Training-Free Variable-Rate KV Cache Compression for Long-Context LLMs arXiv:2607.15498v1 Announce Type: new Abstract: The key-value (KV) cache is the main memory bottleneck in long-context large language model (LLM) inference. Two leading training-free families are both structurally limited: token-selection methods (SnapKV, Ada-KV) score… 31 r/LocalLLaMA community 25d ago Fractale-350M-base: memory as trained behaviour instead of long context, a fully open research release Some of you may remember my post about the research project behind this: a trained fast-weight memory, with the paper and the full research log at github.com/kkuette/thought-bank. This is the follow-up. The first public model of the series is out. Quick context: solo researcher,… 11 Hugging Face Daily Papers research 27d ago LongStraw: Long-Context RL Beyond 2M Tokens under a Fixed GPU Budget Abstract A growing gap separates inference context lengths from RL post-training: inference systems are approaching million-token contexts, while post-training workloads often remain at 256K tokens or below and rely on length generalization at deployment. The gap is especially… 16 arXiv — Machine Learning research 28d ago LongStraw: Long-Context RL Beyond 2M Tokens under a Fixed GPU Budget arXiv:2607.14952v1 Announce Type: new Abstract: A growing gap separates inference context lengths from RL post-training: inference systems are approaching million-token contexts, while post-training workloads often remain at 256K tokens or below and rely on length generalization… 14 arXiv — NLP / Computation & Language research 28d ago PReM: Learning What to Preserve and When to Refresh for Context Compression arXiv:2607.14327v1 Announce Type: new Abstract: Efficient long-context inference is not only about reducing memory cost, but also about keeping useful contextual evidence accessible as generation proceeds. However, existing compression-oriented approaches, such as key-value (KV)… 32 r/LocalLLaMA community 28d ago Anyone else completely tuning out these massive "open weight" drops? Tbh the benchmarks on stuff like GLM-5.2 look insane. 753B params, 1M context, MIT license... everyone is throwing a party on the front page right now. But like... what is actually "local" about this anymore? A 700B+ MoE isn't fitting on anyone's home rig. Even if you absolutely… 29 Smol AI News news-outlet 29d ago not much happened today **Moonshot AI** launched **Kimi K3**, a frontier-class open-weights model with **2.8T parameters**, **1M-token context window**, and **native multimodal input**. It features novel **Kimi Delta Attention (KDA)** enabling up to **6.3x faster decoding** and **Attention Residuals**… 18 r/LocalLLaMA community 29d ago Qwen 3.6 27B is solid up to 262K context. How high have you guys gone above that using Rope/Yarn scaling? Stack: i7 12700K | RTX 3090 TI | 96GB RAM Qwen 3.6 27B Q3/Q5 KXL UD I've been pushing Qwen 3.6 27B above 200K ctx all week, and it handles it like a champ. I'm impressed. Today I hit the ceiling at 262K and it's still functioning well and coherent. I'm planning on trying out… 22 arXiv — Machine Learning research 29d ago Adaptive Filtering of the KV Cache: Diagnosing and Correcting Structural-Role Bias in LLM Inference arXiv:2607.13205v1 Announce Type: cross Abstract: Attention-based KV cache eviction (H2O and its descendants) compresses the memory-constrained state of a long-context model by ranking tokens on accumulated attention mass, treated here as signal energy, and keeping the heaviest.… 11 Vercel — AI dev-tools 29d ago Kimi K3 is now available on AI Gateway Kimi K3 from Moonshot AI is now available on AI Gateway. K3 is an open-source model with a 1M-token context window and native visual understanding, accepting text, image, and video inputs. Built for long-horizon software engineering, knowledge work, and deep reasoning, K3 is… 35 Page 2 of 8 · 378 articles ← Newer Older →
