News / #gpu Tag Gpu 500 articles archived under #gpu · RSS Sign in to follow llama.cpp releases dev-tools 2h ago b10424 dflash : clarify output logging of target_layer_ids ( #27013 ) This commit tries to make the logging of target_layer_ids a bit clearer and easier to read. Currently the output generated looks like this: 0.00.468.624 D load_arch_hparams: DFlash extract_layers = [0.00.468.626 D 2,… 25 arXiv — Machine Learning research 3h ago CAKE: Compiler-Agent Co-Design for Frontier Kernel Evolution arXiv:2608.12629v1 Announce Type: new Abstract: GPU kernel agents and GPU programming languages have advanced separately, leaving expert kernels difficult to reproduce. Agents usually treat the compiler as a fixed black box and receive only errors, correctness outcomes, and… 11 arXiv — Machine Learning research 3h ago A Contract-Grade Verifier for LLM-Generated GPU Kernels, and a Native Blackwell Backward for the Gated-Linear-Recurrence Family arXiv:2608.12700v1 Announce Type: new Abstract: Systems that generate GPU kernels with language models report high correctness rates. Those rates come from a single loose test: run the kernel on a few random inputs at one fixed shape and accept it if the output is close to a… 20 arXiv — NLP / Computation & Language research 3h ago The "Knowledge-Behavior Gap" in Cultural Taboo Safety of Large Language Models arXiv:2608.12341v1 Announce Type: new Abstract: Cultural taboo safety is essential for deploying large language models (LLMs), as culturally insensitive outputs may cause offense or even social harm. However, existing cultural benchmarks primarily assess cultural knowledge or… 11 arXiv — NLP / Computation & Language research 3h ago Novels generated by language models show compressed formal variation arXiv:2608.12630v1 Announce Type: new Abstract: While large language models can generate entire novels, there is little information about the level of formal variation in their output over many generations. Rather than asking whether individual passages can be identified as… 33 arXiv — NLP / Computation & Language research 3h ago Prompts in the Wild: A Large Analyzed Collection of Transactional Prompts in Code arXiv:2608.12905v1 Announce Type: new Abstract: The behavior of contemporary generative Large Language Models (LLMs) is directly shaped by prompts, unstructured texts that describe the desired output and model behavior. In this paper we argue that prompts are linguistic objects… 35 arXiv — NLP / Computation & Language research 3h ago Large Language Models Can Follow Instructions, But Not Many at Once: Phase Transitions in Compositional Constraint Satisfaction arXiv:2608.12426v1 Announce Type: cross Abstract: Large language models are increasingly deployed in settings that require simultaneous adherence to multiple explicit constraints - reasoning structure, safety boundaries, output schemas. Individual constraints are handled… 33 arXiv — NLP / Computation & Language research 3h ago TEMPO: Makespan-Aware Expert-Parallel Load Balancing Across Memory- and Compute-Bound Regimes arXiv:2608.13057v1 Announce Type: cross Abstract: In expert-parallel (EP) MoE serving, every layer synchronizes at the slowest GPU. Dispatchers balance token counts (EPLB, LPLB, UltraEP) or activated-expert counts (METRO), assuming expert time is linear in one. Measurements on… 32 Anthropic SDK (Python) releases dev-tools 13h ago v0.122.0 0.122.0 (2026-08-13) Full Changelog: v0.121.0...v0.122.0 Features api: add output_behavior to dream creation (create a new memory store or update the input store in place) ( 852c4bb ) Bug Fixes bedrock,aws: run SigV4 signing off the event loop in async clients ( #334 ) ( 2bae6c8… 23 TechCrunch — AI news-outlet 16h ago Nvidia’s new $500B plan is risky but brilliant, especially for aging GPUs Nvidia has a plan to make sure its GPUs won't lose value. It wants to convince a new crop of financiers to keep lending for AI buildouts. 30 OpenAI official-blog 21h ago Previewing Ultrafast mode: GPT-5.6 Sol at up to 14X the speed Preview Ultrafast, a new OpenAI API service tier that runs GPT-5.6 Sol up to 14× faster. Powered by Cerebras, it delivers up to 750 output tokens per second. 24 Hugging Face Daily Papers research 22h ago Ready Cohorts: Bounding GPU Opportunity and Avoiding Host Round Trips in LLM-Agent Control Abstract Two studies define measurable GPU control gates for LLM-agent services by analyzing concurrent cohort scheduling and on-device routing versus host redispatch. Generated by thinkingmachines/Inkling-Small LLM-agent services repeatedly execute small deterministic… 25 r/LocalLLaMA community 1d ago Nvidia doubles RTX PRO 6000 Blackwell's MSRP to a staggering $16,000 — 96GB card started pre-orders below $8,000 last year Given Gavin Bakers recent interview where he says: > Multiple private companies are planning on spending at least 2x more per GPU for compute as contracts roll-off and some have spoken about this publicly. https://x.com/i/status/2082166566280642676 How long before DGX Sparks… 20 arXiv — Machine Learning research 1d ago Low-Interaction-Rank Learning: Unifying Multiplicative Dual-Encoder Heads arXiv:2608.11661v1 Announce Type: new Abstract: A multiplicative dual-encoder network computes a real-valued output for a pair of inputs as the inner product of their separate encodings. This architecture has been developed independently in operator learning, bipartite matching,… 26 arXiv — Machine Learning research 1d ago Kernel Methods for Learning Operators with Multiple Inputs and Outputs arXiv:2608.11831v1 Announce Type: new Abstract: Learning mappings between infinite-dimensional objects is a central challenge in scientific machine learning. We introduce a general kernel-based encoder-decoder framework for operator learning that separates observation,… 12 arXiv — Machine Learning research 1d ago A Factor Graph Approach to Scalable Multi-Output Gaussian Process Regression arXiv:2608.11917v1 Announce Type: new Abstract: Multi-output Gaussian process regression scales cubically in the number of observations times outputs, and dense kernel-matrix methods need bespoke handling whenever different outputs are observed at different inputs. We express… 36 arXiv — Machine Learning research 1d ago SoftWater: Class-Aware Rate Allocation for Softmax Quantization arXiv:2608.12026v1 Announce Type: new Abstract: Post-training quantization pipelines routinely leave the softmax output layer in high precision. Yet in small LLMs with modern vocabularies, the head holds 15--30\% of all parameters, so a nominal ``2-bit'' model with an fp16 head… 27 arXiv — NLP / Computation & Language research 1d ago Backtrader-Bench: Benchmarking LLM Agents on Algorithmic Trading with Self-Generated MCQs arXiv:2608.11232v1 Announce Type: new Abstract: Evaluating LLM coding agents in algorithmic trading is difficult because static benchmarks risk data contamination and numerical backtest outputs require ground truth from actual code execution. We present Backtrader-Bench, a… 18 arXiv — NLP / Computation & Language research 1d ago Measure, Don't Optimize: Forecasting Recovery in LLM Unlearning arXiv:2608.11408v1 Announce Type: new Abstract: Prior white-box studies show that large language models can retain latent traces of target knowledge after unlearning, even when the knowledge is no longer expressed in their outputs. However, existing audits remain limited to… 25 arXiv — NLP / Computation & Language research 1d ago Is Convergence Inevitable? Tracing Output Homogeneity Back to Base Models arXiv:2608.11426v1 Announce Type: new Abstract: The lack of diversity in LM content is widely attributed to the alignment process, but how and where exactly in the pipeline this collapse begins is unknown. We argue that output homogeneity is likely learned during the pretraining… 12 arXiv — NLP / Computation & Language research 1d ago Beyond Single-Turn Confidence: Trajectory-Adapted Uncertainty Quantification for LLM Agents arXiv:2608.11552v1 Announce Type: new Abstract: Uncertainty quantification (UQ) methods for language models are typically evaluated on single-turn outputs, where uncertainty is attached to one generated answer. For LLM agents, however, the unit of observation is an interactive… 4 arXiv — NLP / Computation & Language research 1d ago Locating and Controlling Implicit Personalization in Large Language Models arXiv:2608.11735v1 Announce Type: new Abstract: Large language models (LLMs) often shift their outputs in response to implicit demographic cues even when users never state a demographic identity. Previous work has documented this behavior, but the connection between these… 8 arXiv — NLP / Computation & Language research 1d ago LazyTrain: Limited-resource Allocation toward Zero-waste Yield Optimization in Large Language Model Training arXiv:2608.11919v1 Announce Type: new Abstract: Training large language models on limited hardware is increasingly a scheduling problem across GPU compute, host memory, PCIe transfer, and storage bandwidth. Existing offloading systems reduce GPU residency, and MegaTrain shows… 19 r/LocalLLaMA community 1d ago Qwen3.6 35B (2 min) vs Muse Glimmer 30B (4 min) on custom Llama.cpp build (RTX 5080) Muse Glimmer 30B feels significantly more precise and reliable, it almost never drops the ball or breaks rules. However, its designs lack creative depth and richness. Qwen3.6 35B, on the other hand, is prone to more occasional blunders/hallucinations, but its creative output is… 35 Ollama releases dev-tools 1d ago v0.32.10-rc0: nn: speed up prefill on double-scale nvfp4 models ModelOpt checkpoints apply a float32 global scale to every projection output on top of the per-group quantization scales. Running the multiply and the cast back to the activation dtype as separate eager ops costs an extra kernel launch and a materialized intermediate per… 14 NVIDIA Developer Blog official-blog 1d ago Serve Qwen3.8-2.4T-A95B, a 2.4T-Parameter Model, with Configurable Reasoning on NVIDIA GB300 NVL72 Alibaba released the open weights for Qwen3.8-2.4T-A95B (Qwen3.8-Max), its largest open-weight model, bringing near-frontier capabilities to the open... 35 NVIDIA Developer Blog official-blog 1d ago How to Choose Full-Stack Observability for NVIDIA AI Factories AI infrastructure spans multiple layers, from compute and networking to storage, orchestration, and applications. When performance degrades, identifying the... 34 r/LocalLLaMA community 1d ago Best models 14b and smaller as of today? For the GPU impoverished   submitted by   /u/Thatisverytrue54321 [link]   [comments] 10 r/LocalLLaMA community 1d ago NVIDIA's Fastest Blackwell GPU, the 96 GB RTX PRO 6000, Now Costs $16,000, Almost Double Its Original Price   submitted by   /u/ab2377 [link]   [comments] 6 r/LocalLLaMA community 1d ago According to AMD, Arm, and Microsoft, agentic AI could push CPU-to-GPU ratios from 1:4 to even1:1 In OCP APAC 2026, Tai AMD SVP of compute and enterprise AI said agents don't cut GPU demand but they just pile on a whole extra layer of orchestration, retrieval, and tool-calling work that runs on CPUs instead And the usual 1:4 CPU-to-GPU ratio could move toward 1:2 or even 1:1… 23 Hugging Face Daily Papers research 2d ago Articulated Object Reconstruction from Rest-State Observation Abstract A rest-state framework reconstructs articulated objects from a single closed configuration by fusing vision-language outputs into consistent part meshes and validating synthesized motion hypotheses via geometric consistency. Generated by thinkingmachines/Inkling-Small… 29 r/LocalLLaMA community 2d ago RTX 6000 PRO price raised to $16,000 USD on the Nvidia website   submitted by   /u/Norwood_Reaper_ [link]   [comments] 29 arXiv — Machine Learning research 2d ago Fisher8: Stabilizing Neural Heteroscedastic Regression via Output-Layer Fisher Geometry arXiv:2608.10374v1 Announce Type: new Abstract: Training neural networks to jointly predict mean and uncertainty estimates from noisy observations can be unstable, prompting a series of independent stabilization efforts. We argue that these interventions highlight a common… 16 arXiv — Machine Learning research 2d ago ProbGuard: Calibrated Safety Risk Estimation from LLM Output Distributions arXiv:2608.10621v1 Announce Type: new Abstract: Recent research on Large Language Model (LLM) safety has widely adopted guardrails to identify unsafe LLM outputs. Existing guardrails typically formulate safety assessment as a deterministic classification task, mapping a discrete… 37 arXiv — NLP / Computation & Language research 2d ago Mapping and Measuring the Behavioral Evolution of Large Language Models arXiv:2608.11027v1 Announce Type: cross Abstract: Benchmark leaderboards summarize how well a language model performs, but not how its behavior relates to that of other models or changes across generations. We characterize the output behavior of 32 models from six families using… 27 arXiv — NLP / Computation & Language research 2d ago The Parser Already Knows: Lightweight Bias Correction in Constrained Decoding arXiv:2608.10137v1 Announce Type: new Abstract: Grammar Constrained Decoding (GCD) forces Language Models (LMs) to produce syntactically valid outputs by masking out non-conforming tokens at each step. However, rigid masking distorts the model's underlying probability… 16 arXiv — NLP / Computation & Language research 2d ago Mitigating Context Interference for Reliable and Efficient Search Agents arXiv:2608.10743v1 Announce Type: new Abstract: Recent research empowers Large Language Models (LLMs) as multi-turn search agents to iteratively retrieve and generate outputs until complex tasks are solved. However, the contexts of multi-turn search agents are lengthy and… 38 arXiv — NLP / Computation & Language research 2d ago What Iterated Self-Feeding Probes of Language Models Measure, and a test that separates the construction from the model arXiv:2608.10986v1 Announce Type: new Abstract: A growing class of methods probes a language model by feeding it its own output: self-consistency, iterated refinement, agentic loops. We ask what such a probe measures, in a construction chosen to make the question sharp: a ring… 5 arXiv — NLP / Computation & Language research 2d ago Attention-Path Fragility as an Uncertainty Signal in Large Language Models arXiv:2608.11138v1 Announce Type: new Abstract: We propose that a model's uncertainty about a token is reflected not only in the breadth of its output distribution but also in whether a confident prediction is \emph{fragile} under perturbation of its attention pathways. We… 27 arXiv — NLP / Computation & Language research 2d ago Withholding the Completing Chunk: Deterministic Pair-Completion Guardrails for Streaming LLM Output arXiv:2608.10279v1 Announce Type: cross Abstract: Streaming language-model output creates a release-timing problem: complete-response moderation acts after streamed text has escaped, whereas repeated semantic classification of partial text can be costly and unstable. We study a… 21 Hacker News — AI on Front Page community 2d ago Nvidia Nemotron 3.5 Lightning and NeMo Switchyard Article URL: https://blogs.nvidia.com/blog/nemotron-lightning-switchyard-rtx-dgx/ Comments URL: https://news.ycombinator.com/item?id=49263340 Points: 220 # Comments: 113 26 Hugging Face Daily Papers research 2d ago Gaming Without an Attacker: Benchmark Fingerprinting in LLM-Driven Search Under Selection Pressure Abstract Optimized GPU kernel benchmarks reveal that evolutionary LLM proposals exploit evaluation configurations, causing widespread failure to generalize to held-out settings. Generated by thinkingmachines/Inkling-Small Benchmarks for systems that are optimized against the… 11 NVIDIA Developer Blog official-blog 2d ago NVIDIA JetPack 7.2.1 Adds Agentic Video Skills and T3000 Emulation Video is a core data path across NVIDIA Jetson applications, from robotics and intelligent video analytics to industrial automation, healthcare, media... 5 r/LocalLLaMA community 2d ago Muse Glimmer 30B running locally in-browser with custom WebGPU kernels at ~25 tok/s on an M4 Max (same speed as llama.cpp)   submitted by   /u/xenovatech [link]   [comments] 23 r/LocalLLaMA community 2d ago nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16   submitted by   /u/TheRealMasonMac [link]   [comments] 21 Hacker News — AI on Front Page community 2d ago Apple Silicon and macOS VMs: 11–16× Faster LLM Inference with Llama.cpp Article URL: https://github.com/trycua/cua/blob/main/blog/gpu-passthrough-macos-vms.md Comments URL: https://news.ycombinator.com/item?id=49259339 Points: 211 # Comments: 32 26 r/LocalLLaMA community 2d ago nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 · Hugging Face   submitted by   /u/coder543 [link]   [comments] 25 NVIDIA Developer Blog official-blog 2d ago NVIDIA Nemotron 3.5 Lightning Delivers Fast, Accurate Specialized Task Execution for Long-Running Agents Long-running AI agents spend most of their time on high-volume execution: tool calls, result validation, and subagent delegation. Using a frontier reasoning... 34 NVIDIA Developer Blog official-blog 2d ago Route AI Agent Workloads Across Models with NVIDIA NeMo Switchyard Building an AI agent does not end with choosing a single model. Each model has its own strengths, weaknesses, and cost profile, which can shift from one... 4 r/LocalLLaMA community 2d ago Revision Prompting: Trades slow (decoded) output tokens for cheap (prefilled) input tokens. TL;DR: If you re-run the same prompt whenever the input changes, try sending the old input/output plus a diff of the input, and ask the model for a patch to the output. You generate ~2-10x fewer output tokens, and the untouched parts of the output stay byte-identical. This… 22 Page 1 of 10 · 500 articles Older →
