News / #gpu Tag Gpu 500 articles archived under #gpu · RSS Sign in to follow arXiv — Machine Learning research 23d ago BRIDGE: Bottleneck-Aware Regulator-Set Inference and Diagnosis for Cooperative Gene Regulatory Recovery arXiv:2607.18602v1 Announce Type: new Abstract: Cooperative gene regulation often depends on groups of regulators acting jointly, but most gene regulatory network (GRN) inference methods output pairwise regulator-target rankings. We introduce Bottleneck-Aware Regulator-Set… 37 arXiv — NLP / Computation & Language research 23d ago Mark, Don't Erase: Token Inoculation for Dual-Use Knowledge in LLMs arXiv:2607.18639v1 Announce Type: cross Abstract: Safety interventions on dual-use knowledge typically choose between destroying hazardous content (e.g., unlearning, filtering) and suppressing it at the output layer (e.g., refusal training); both pay a tax in adjacent-domain… 34 arXiv — Machine Learning research 23d ago GEqTrain: A Configuration-Driven Framework for Retargeting Equivariant Graph Neural Networks Across 3D Scientific Tasks arXiv:2607.19083v1 Announce Type: new Abstract: Equivariant graph neural networks provide a powerful modeling language for three-dimensional scientific data, but their reuse is often limited by implementations tied to specific tasks, outputs, and training regimes. We present… 17 arXiv — NLP / Computation & Language research 23d ago Structured Output Collapses Answer Diversity Across 44 Language Models arXiv:2607.18476v1 Announce Type: new Abstract: When a language model must choose one answer from a large space of equally valid options, a format clause -- "Reply with JSON only" -- changes which answer it chooses. We re-run the One-Word Census (arXiv:2607.12796): 31… 9 arXiv — NLP / Computation & Language research 23d ago Evaluating Style-Personalized Text Generation: Challenges and Directions arXiv:2508.06374v3 Announce Type: replace Abstract: With the surge of large language models (LLMs) and their ability to produce customized output, style-personalized text generation--"write like me"--has become a rapidly growing area of interest. However, style personalization… 14 llama.cpp releases dev-tools 23d ago b10088 llama-arch: fix DeepSeek4 APE tensor op ( #25945 ) 4 llama.cpp releases dev-tools 23d ago b10079 server : properly handle null llama_context ( #25868 ) Co-authored-by: Stanisław Szymczyk [email protected] Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu… 11 llama.cpp releases dev-tools 23d ago b10078 vulkan: Refactor vk_queue to use per-instance mutexes and unique handles ( #23570 ) Refactor vk_queue to use per-instance mutexes and unique handles integrates VK_KHR_internally_synchronized_queues, abstracting the queue submission into a polymorphic interface that completely… 31 r/MachineLearning community 23d ago Looking for feedback on my GPU-accelerated Snake AI project [P] I've been building an AI that learns to play the classic Snake game through reinforcement learning. The goal is to reach high scores while keeping training time as low as possible. The current version averages 86 points (87 is the maximum) after less than 10 hours of training on… 31 llama.cpp releases dev-tools 23d ago b10077 ggml-openvino: Add GGML_BACKEND_DL_IMPL invocation for OpenVINO backend ( #25795 ) This adds the missing GGML_BACKEND_DL_IMPL() macro invocation, that other backends have. Fixes #25586 for me Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon… 6 Hugging Face official-blog 23d ago The State of Simulation for Physical AI: An Overview Back to Articles a]:hidden"> The State of Simulation for Physical AI: An Overview Enterprise + Article Published July 21, 2026 Upvote - Johnny Nuñez Cano johnnynv nvidia Mitesh Patel mitp nvidia Asier Arranz asiernvidia nvidia lior ben horin liorbenhorin-nv nvidia Raymond Lo… 23 llama.cpp releases dev-tools 23d ago b10076 CUDA: vectorize same-type get_rows with int4 copy ( #25929 ) k_get_rows_float did a scalar one-element-per-thread copy and recomputed the row-invariant work (index load, fast_div_modulo, src/dst row pointers) for every element. Hoist that out of the per-element loop, and add a… 26 NVIDIA Developer Blog official-blog 23d ago NVIDIA Vera CPU: Olympus Cores Built for Maximum Single-Thread Performance in Agentic AI Agentic AI shifts more of the critical execution path onto the CPU. Agents operate in sandboxes to execute code, invoke tools, retrieve context, interact with... 38 NVIDIA Developer Blog official-blog 23d ago Inside NVIDIA Rubin GPU Architecture: Powering the Era of Agentic AI What began as discrete AI model training and human-facing chat interfaces has evolved into always-on AI factories dedicated to producing intelligence at scale.... 6 NVIDIA Developer Blog official-blog 23d ago Setting a World Record for MoE Pre-Training on NVIDIA GB300 NVL72 Frontier model pre-training has converged on mixture of experts (MoE), which is fundamentally changing what limits large-scale AI training. As compute per token... 23 Hugging Face Daily Papers research 24d ago Apple-π: Benchmarking Thinking with Video Towards Law-Grounded Physical Intelligence Abstract Modern video generation models are increasingly hailed as emerging world models with an internalized grasp of physical law. Yet existing benchmarks largely evaluate physical plausibility only at the output level, without verifying whether the model arrives there through… 16 arXiv — Machine Learning research 24d ago BACON: Budgeted Human Calibration for Modeling and Evaluation with Multiple AI Judges arXiv:2607.16239v1 Announce Type: new Abstract: AI judges offer a scalable, low-cost alternative to human evaluation, but their outputs can be biased relative to human preferences and highly item-dependent, varying across judges, tasks, and domains. When uncalibrated AI… 14 arXiv — Machine Learning research 24d ago KernelBench-Verified: Do LLM-Generated Kernels Actually Beat PyTorch? arXiv:2607.16241v1 Announce Type: new Abstract: Recent large language models (LLMs) can generate custom CUDA kernels that appear to outperform PyTorch on benchmarks such as KernelBench. Building upon this foundational framework, we demonstrate that frontier models frequently… 36 arXiv — NLP / Computation & Language research 24d ago Diagnosing Correctness Probes under Self-Judgement Confounding arXiv:2607.16799v1 Announce Type: new Abstract: Hidden-state readouts can predict whether language-model outputs are correct, but objective correctness (OC) usually agrees with the model's own self-judgement (SJ), leaving the decoded signal semantically ambiguous. We construct… 13 arXiv — NLP / Computation & Language research 24d ago AI_LectureNote: A Retrospective Pilot Study of a Post-ASR Workflow for English-Script Rendering and Semantic Drift in Korean-English Medical Lectures arXiv:2607.17237v1 Announce Type: new Abstract: AI_LectureNote is a historical, readability-oriented post-ASR workflow for Korean-English medical lectures. It rewrites speech-to-text output into study transcripts while restoring Latin-script medical terms rather than Korean… 5 arXiv — NLP / Computation & Language research 24d ago Zero Hallucination, by Construction: Hallucination-Aware Layered Oversight for Trustworthy Enterprise AI arXiv:2607.17883v1 Announce Type: new Abstract: Enterprises will not deploy AI agents they cannot trust, and the most-cited reason for distrust is hallucination: confident, fluent output that is simply not true. The common response is to wait for a model that does not… 21 arXiv — NLP / Computation & Language research 24d ago RAIL Guard: Closing the Evaluation-to-Remediation Gap in Responsible AI for LLM Agents arXiv:2607.16215v1 Announce Type: cross Abstract: Existing guardrail systems for large language model agents operate as binary classifiers that block unsafe content, leaving organizations to discard failing outputs and retry from scratch. We introduce RAIL Guard, a closed-loop… 34 arXiv — NLP / Computation & Language research 24d ago SlotGuard: Stop Oversharing Private Local Context in LLM Agent Transcri arXiv:2607.17147v1 Announce Type: cross Abstract: LLM agents can leak privacy (e.g., paths, emails) and credentials (e.g., API keys) as agent observations (e.g., tool outputs, shell logs, and file reads) are appended to provider-bound transcripts. Existing placeholder redaction… 4 Vercel — AI dev-tools 24d ago Gemini 3.6 Flash and Gemini 3.5 Flash-Lite are now available on AI Gateway Gemini 3.6 Flash and Gemini 3.5 Flash-Lite are now available on AI Gateway. Gemini 3.6 Flash improves quality across coding, agentic tasks, and web development while consuming fewer tokens and making fewer model calls. It produces cleaner web and app development output. Gemini… 6 llama.cpp releases dev-tools 24d ago b10075 hexagon: add CLAMP op ( #25934 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu… 19 r/LocalLLaMA community 24d ago Running a 13M ASR conformer on a microcontroller Hello everyone, I wanted to share a recent project of mine, which brings a 13.1 million parameter convolution transformer model to a < $10 microcontroller (more specifically, the ESP32-S3). It's a distilled and quantized version of nvidias small conformer model from huggingface.… 38 Hugging Face official-blog 24d ago Introducing Cosmos 3 Edge Back to Articles a]:hidden"> Introducing Cosmos 3 Edge Enterprise + Article Published July 20, 2026 Upvote - Pranjali Joshi PranjaliJoshi nvidia Saeed Babamohamadi SaeedBabamohamadi nvidia The real world is vast and to operate in it physical AI systems need to understand how a… 32 NVIDIA Developer Blog official-blog 24d ago NVIDIA NVLink: The Scale-Up Network for AI Factories The demand for AI continues to accelerate. Workloads are getting larger, models are becoming more complex, and there is mounting pressure to deploy AI compute... 12 NVIDIA Developer Blog official-blog 24d ago Integrate NVIDIA Omniverse RTX Sensor Simulation Into Existing Apps Developers building 3D, design, simulation, robotics, and industrial digital twin applications need ways to bring physical AI capabilities into the tools and... 12 r/LocalLLaMA community 24d ago Unsloth now supports AMD! Hey r/LocalLLaMA folks! Unsloth now officially supports AMD hardware for local inference, fine-tuning, reinforcement learning, and deployment! It's been in the works for quite some time, but it works on Windows, Linux & WSL devices (+ technically Mac) with AMD GPUs! Unsloth… 24 r/LocalLLaMA community 24d ago Our 1-bit quant of Hy3 295B runs 2.2x faster than the cloud API with no quality loss We quantized Tencent's Hy3 295B down to 1 bit and got a 92GB IQ1_M GGUF, small enough for one 4-GPU box. We ran it on 4x RTX 5090 against the same Hy3 over the cloud API. Both got the same one-shot task. Each model built a self-playing retro game in one HTML file. We ran three… 36 llama.cpp releases dev-tools 25d ago b10069 opencl: Support broadcast for Adreno MUL_MAT and honor view_offs for Adreno Q8_0 MUL_MAT for llama-server multi-stream ( #25910 ) opencl: handle broadcast for adreno gemm/gemv_noshuffle opencl: honor view_offs for adreno noshuffle gemm/gemv opencl: general GEMM/GEMV support… 14 arXiv — Machine Learning research 25d ago ASK-NN: An Asymmetric Nearest-Neighbor Test that detects Distribution Drifts in Natural Language arXiv:2607.15607v1 Announce Type: new Abstract: Hallucinations and artificial text in LLM-generated outputs often appear as distributional deviations between prompt and response hidden-state distributions. Since prompts or retrieved contexts typically serve as reference samples… 6 arXiv — Machine Learning research 25d ago (MPO)$^2$: Multivariate Polynomial Optimization based on Matrix Product Operators arXiv:2607.15916v1 Announce Type: new Abstract: Central to machine learning and signal processing is the ability to perform universal function approximation and learn complex input-output relationships from limited numbers of observations. Multivariate polynomial models offer a… 5 arXiv — Machine Learning research 25d ago PagedWeight: Efficient MoE LLM Serving with Dynamic Quality-Aware Weight Quantization arXiv:2607.16184v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) is a popular class of large language models (LLMs), offering high efficiency and accuracy. However, in KV-cache-intensive serving scenarios, MoEs often exhibit a tension between the GPU memory requirements… 19 arXiv — Machine Learning research 25d ago Beyond a Single Direction: Chain-of-Thought Disrupts Simple Steering of Refusal arXiv:2605.26772v1 Announce Type: cross Abstract: Large reasoning models (LRMs) generate chain-of-thought (CoT) traces before producing final outputs, introducing a dynamic internal state that may complicate control mechanisms such as refusal. Unlike instruction-tuned LLMs,… 26 arXiv — NLP / Computation & Language research 25d ago Latent Fusion Jailbreak: Blending Harmful and Harmless Representations to Elicit Unsafe LLM Outputs arXiv:2508.10029v3 Announce Type: replace Abstract: Safety-aligned large language models can still be manipulated through white-box interventions that modify their internal representations. We introduce Latent Fusion Jailbreak (LFJ), which works by pairing a harmful query with a… 31 arXiv — NLP / Computation & Language research 25d ago Speculative Decoding with a Speculative Vocabulary arXiv:2602.13836v2 Announce Type: replace Abstract: Speculative decoding has rapidly emerged as a leading approach for accelerating language model (LM) inference, as it offers substantial speedups while yielding identical outputs. This relies upon a small draft model, tasked… 16 Vercel — AI dev-tools 25d ago Configure where run state lives in Vercel Workflows Vercel Workflows now keeps each run's state, queue dispatch, and output streams in a single home region: the region where the run starts by default, or any target region you choose. A run keeps its home region for its lifetime, so for agents built on Workflows, the whole loop… 17 r/LocalLLaMA community 25d ago [Paper] Automated Tensor Scheduling for Hybrid CPU-GPU LLM Inference on Consumer Devices Running large language models on consumer devices such as laptops and desktops is challenging because model weights often exceed GPU memory capacity, making offloading inference necessary to extend effective model capacity with CPU memory. Existing offloading systems, however,… 26 r/LocalLLaMA community 25d ago moonshot runs out of GPU For the first time, a Chinese artificial intelligence company has run out of GPU capacity. Moonshot AI has decided to suspend new subscriptions and eliminate free access for users.   submitted by   /u/Illustrious-Swim9663 [link]   [comments] 12 r/LocalLLaMA community 26d ago How do we benefits from 2+ T models? Hey, I’ve been really excited to see the latest models being released, but I keep wondering: what are we actually supposed to do with them? I have 4× RTX 6000 Max-Q GPUs, 7× RX 7900 XTXs, 5× modded 48GB RTX 4090s, and a lot of DDR5 RAM.... and honestly, I can’t even imagine… 15 r/LocalLLaMA community 26d ago poor man's way to local inference on the go Many bring egpu to game on laptop, yet here I am fiddling with llama cpp params for 1-time crappy HW configuration for Qwen3.6 35B A3B. idk if I'm having fun or not, but running llama bench runs are surely a good way to kill some time, I guess p.s. I really like recently added… 22 r/LocalLLaMA community 26d ago Deepseek v4 Flash on 80 GB VRAM and 128 GB DDR4 RAM I am using unsloth Q8 Deepseek v4 Flash. So far I am able to run properly with the following command CUDA_DEVICE_ORDER=PCI_BUS_ID CUDA_VISIBLE_DEVICES=0,2,1 \ llamacpp/llama.cpp/build/bin/llama-server \ --model… 33 r/LocalLLaMA community 26d ago Qwen vs Gemma Hi! Been doing some local LLM stuff, and I can't help but notice: despite vastly-superior benchmark scores, Qwen 3.6 35a3B feels... substantially less intelligent than Gemma 4 26a4B (QAT). In terms of prompt adherence, output coherence, and just general "sanity", Gemma seems… 4 r/LocalLLaMA community 26d ago CMP 170HX 8gb — Perf + Memory + PCIe Gen2 Unlock - NVIDIA Driver 610.43.03 (patched open kernel modules) some proof: https://i.imgur.com/qqEjxmW.png https://github.com/amoghmunikote/cmpunlocker CMP 170HX 8gb — Perf + Memory + PCIe Gen2 Unlock NVIDIA Driver 610.43.03 (patched open kernel modules) GPU: CMP 170HX (0x20C2) Result: 64 GB HBM2e + 173 TFLOPS BF16 + PCIe Gen2 x4 (2 GB/s)… 38 r/LocalLLaMA community 26d ago How are y’all stomaching the “AI Boom” prices? I am in the middle of considering an upgrade to my Home Server. I want to get a decent GPU for LocalAI. I mean, I have an RTX 3060 TI 16GB, I know that is much more than most people have, but even though it has a lot of cram the bus width is really slowing it down - I get ~23… 26 llama.cpp releases dev-tools 27d ago b10068 model: rotate injected K/V cache for DFlash ( #25823 ) dflash: rotate injected K/V cache when using K/V quantization Update src/models/dflash.cpp Co-authored-by: Georgi Gerganov [email protected] clearer format remove trailing whitespace Co-authored-by: Georgi Gerganov… 21 llama.cpp releases dev-tools 27d ago b10067 llama-quant : exclude i32 ffn_gate_tid2eid routing table from quantization ( #25787 ) DeepSeek-V4's ffn_gate_tid2eid tensor is an i32 token-id -> expert-id index table, not weights. It was never added to the name-based exclusion list alongside ffn_gate_inp.weight, so… 20 r/LocalLLaMA community 27d ago SigLIP 2 text embedding on CPU with Rust + ONNX We’re building a robotics data platform with a lot of images, video, and text metadata. For search, we use SigLIP 2. GPUs handle batched asynchronous image/video embedding and indexing, while this small Rust + ONNX Runtime service handles live text queries on CPU. Both land in… 37 Page 8 of 10 · 500 articles ← Newer Older →
