News / #gpu Tag Gpu 500 articles archived under #gpu · RSS Sign in to follow arXiv — Machine Learning research 29d ago VAIOM: Continuous-Input, Discrete-Output Decoder-Only Financial Sequence Modeling arXiv:2607.13929v1 Announce Type: new Abstract: Financial observations are continuous, heterogeneous, and noisy, whereas decoder-only next-token models are usually built around discrete symbolic inputs. We introduce Vector-Input Autoregressive Inference for Ordinal-Return… 4 arXiv — Machine Learning research 29d ago Improving Wind and Solar Power Prediction with Efficient Wrapper-based Feature Selection: An Empirical Study arXiv:2607.14024v1 Announce Type: new Abstract: With rising global energy demand and growing awareness of climate change and its impacts, the share of renewable energies in the global energy mix continues to grow. Unlike conventional power generation, the output of renewable… 19 arXiv — Machine Learning research 29d ago Text2Sign: A Single-GPU Diffusion Baseline for Text-to-Sign Language Video Generation arXiv:2607.13164v1 Announce Type: cross Abstract: Sign language is a primary communication channel for millions of Deaf and hard-of-hearing people, yet text-to-signer video generation remains costly because video diffusion models are expensive to train and evaluate. This paper… 11 r/LocalLLaMA community 29d ago PSA: Nvidia's CMP 170HX Full Compute and Memory(80GB) may be unlockable via exploit If you don't know, the CMP 170HX is essentially a A100 that has had it's compute and memory crippled so it can only mine crypto. It was a product of the crypto craze, and was released shortly before the crypto crash. Well, I was scrolling around and found out, apparently, it can… 27 r/LocalLLaMA community 29d ago AMD ROCm 7.14 "TheRock" tech preview tagged for latest AMD GPU compute stack   submitted by   /u/Fcking_Chuck [link]   [comments] 33 NVIDIA Developer Blog official-blog 29d ago Build a Multi-Camera 3D Tracking Application with NVIDIA DeepStream 9.1 Skills Developers building video analytics applications across large spaces must track the same object as it moves between camera views. Single-camera 2D tracking... 12 r/LocalLLaMA community 29d ago Inkling by Thinking Machines is the #1 US open weight model now Inkling by Thinking Machines Lab is a huge step forward for US open weight models to catchup w/ China. Inkling solidly beats all US open models including NVIDIA Nemotron Ultra and ranks ~#5 of all open weight models. Congrats to the thinking team!   submitted by  … 37 llama.cpp releases dev-tools 29d ago b10034 opencl: exclude some moe kernels on Adreno a7x ( #25698 ) opencl: exclude Adreno A7x from using Adreno MoE kernels Some compilers for A7x devices miscompile the repack kernels, corrupting the weights and causing MoE models to generate garbage output opencl: exclude A6x and… 22 llama.cpp releases dev-tools 29d ago b10032 cuda : CUDA GGML_OP_LIGHTNING_INDEXER implementation (generic vector kernel + wmma kernel) ( #25545 ) cuda : CUDA GGML_OP_LIGHTNING_INDEXER implementation (generic vector kernel + wmma kernel) chore : remove indentation of #pragma unroll cuda : remove unnecessary kernel template… 37 r/LocalLLaMA community 29d ago Google is updating Gemma 4's chat templates, bringing major fixes to tool calling and reducing "laziness", and enabling Flash Attention 4 on Hopper GPUs, plus an interactive guide on how to work with and improve its vision! Nvm ignore the image links here is the source: https://x.com/googlegemma/status/2077449152062247219 https://huggingface.co/spaces/google/gemma4_vision_token_budget   submitted by   /u/Iwaku_Real [link]   [comments] 16 llama.cpp releases dev-tools 29d ago b10031 tokenize : drop --stdin mutual-exclusion check ( #25672 ) match cli and completion, which don't enforce it macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU)… 13 NVIDIA Developer Blog official-blog 29d ago Building Faster Cryptography with Carryless Multiplication in NVIDIA CUDA 13.3 For over fifteen years, x86 CPUs have shipped with a dedicated hardware instruction for carryless multiplication. It’s a small but stubborn primitive that... 27 llama.cpp releases dev-tools 29d ago b10025 cuda : relax tensor contiguity requirements for quantized concat ( #25678 ) cuda : relax tensor contiguity requirements for quantized concat tests : add test cases for non-contiguous quantized concat ggml : relax contiguity requirements for quantized concat Co-authored-by:… 22 r/LocalLLaMA community 29d ago The best model is the one you can actually run Don't get me wrong, all the big models are amazing, and every contribution to open source models is great. But I'm GPU poor and I can't use them locally. I'm currently running gemma-4-12b-it-qat-GGUF:UD-Q4_K_XL as my personal chat assistant, and I am so so happy with it! I still… 10 Hacker News — AI on Front Page community 1mo ago Running Gemma 4 26B at 5 tokens/sec on a 13-year-old Xeon with no GPU Article URL: https://www.neomindlabs.com/2026/06/08/running-gemma-4-26b-at-5-tokens-sec-on-a-13-year-old-xeon-with-no-gpu/ Comments URL: https://news.ycombinator.com/item?id=48922434 Points: 223 # Comments: 147 10 llama.cpp releases dev-tools 1mo ago b10021 DeepseekV4: reduce graph splits ( #25702 ) macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan)… 9 r/MachineLearning community 1mo ago PyTorch model running 170x slower on T4 vs A100. What could cause a bottleneck this extreme? [D] Hey everyone, Seeing a ~170× slowdown running a point-tracking model on an NVIDIA T4 compared to an A100. On A100 the tracker takes ~0.5 seconds per half-video. On T4 the same call takes ~85 seconds. Video is 47 frames at 256×256, batch 1. I expect a meaningful gap between these… 14 llama.cpp releases dev-tools 1mo ago b10020 sycl : fix get_rows Q2_K, Q4_K, Q5_K ( #25656 ) macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan)… 35 llama.cpp releases dev-tools 1mo ago b10019 sycl : support kernel type fp16 for conv2d_dw ( #25653 ) macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64… 21 r/LocalLLaMA community 1mo ago Bonsai-27B & Ternary-Bonsai-27B - Updates (on PRs) Below Upstream Status sections are from https://github.com/PrismML-Eng/Bonsai-demo Upstream Status for Binary Q1_0 is supported out of the box in upstream llama.cpp across many backends: CPU (generic, NEON, and optimized x86), Metal, CUDA, and Vulkan. Runtime Status llama.cpp… 29 llama.cpp releases dev-tools 1mo ago b10018 sycl : implement xielu op ( #25550 ) macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64… 34 llama.cpp releases dev-tools 1mo ago b10017 sycl: Increase minimum buffer size for USM system allocations ( #25525 ) Raise the threshold for minimum buffer size from 1 GiB to 4 GiB, based on real-world experiments of overcommitting device memory with model weights larger than available VRAM, for example Qwen3.5-35B-A3B-Q8… 28 llama.cpp releases dev-tools 1mo ago b10016 [SYCL] Flash Attention with XMX engine via oneDNN ( #25222 ) [SYCL] F16 (default) Flash Attention with XMX engine via oneDNN graph API; Qwen3.6-27b-Q8_0 prefill speed up x1.21 at p=512 and x4.26 at p=80k [SYCL] Address review on FA oneDNN path. Result: llama-bench---pp512; 32%… 8 arXiv — Machine Learning research 1mo ago Self-Evolving In-Context Learning for Direct Pilot-to-Beamformer Design in MU-MISO Systems arXiv:2607.11970v1 Announce Type: new Abstract: We develop an enhanced in-context learning (ICL) framework to improve the performance of pilot-based beamforming in multi-user multiple-input single-output (MU-MISO) systems. The proposed scheme integrates the ICL-Transformer… 21 arXiv — Machine Learning research 1mo ago Forgetful Attention: A Trainable Support-Vector Memory with Certified Selection and Exact Unlearning arXiv:2607.12204v1 Announce Type: new Abstract: Attention can be viewed as an online learner over context, yet existing test-time memories cannot certify that dropping a token leaves outputs unchanged or delete its influence outright. We introduce Support Vector Attention… 34 arXiv — Machine Learning research 1mo ago Evidence-Grounded Verified Agentic Reasoning: A Path Toward Eliminating LLM Hallucination in Empirical Inference via Tool-Attested Kernel Proofs arXiv:2607.12650v1 Announce Type: new Abstract: Tool access alone does not make LLM empirical reasoning governable: accepted outputs need not descend from attested evidence, and accepted deductions need not hold up under formal scrutiny. We present EG-VAR (Evidence-Grounded… 19 arXiv — Machine Learning research 1mo ago Do You Remember? Toward Memory-Centric Multimodal AI arXiv:2607.11919v1 Announce Type: cross Abstract: Human memory is reconstructive, not a faithful recording. Current multimodal LLMs (MLLMs) lack this capability: they process images through a frozen visual encoder, produce a one-shot text output, and discard internal… 36 arXiv — NLP / Computation & Language research 1mo ago Token Reduction Is Not Cost Reduction arXiv:2607.12161v1 Announce Type: new Abstract: Context-reduction layers for API-based coding agents, including command-output compressors, retrieval rankers, and payload-optimizing proxies, are usually evaluated by how much text they remove. We ask instead: when does reducing… 19 arXiv — NLP / Computation & Language research 1mo ago RCWT: Measuring Task-Budget Displacement from Coordination Content in LLM Calls arXiv:2607.12216v1 Announce Type: new Abstract: Multi-agent and memory-augmented LLM systems often place coordination content, shared state, prior discussion, tool outputs, summaries, and role instructions, inside the same finite prompt used for the current task. This creates a… 31 arXiv — NLP / Computation & Language research 1mo ago Can LLMs Write Reliable Rubrics? A Meta-Evaluation for Experiment Reproduction arXiv:2607.12835v1 Announce Type: new Abstract: Rubric-based evaluation is a promising approach for assessing open-ended outputs from LLM-based research agents, particularly in paper reproduction, where direct paper-to-repository comparison is prone to hallucination. However,… 18 arXiv — NLP / Computation & Language research 1mo ago GRID: Grammar-Railed Decoding for Enterprise SQL Generation arXiv:2607.11951v1 Announce Type: cross Abstract: Large language models can write SQL, but enterprise deployment demands more than plausible text: outputs must be syntactically valid, must respect per-role and per-schema policy, must carry provable (not best-effort) guarantees,… 17 arXiv — NLP / Computation & Language research 1mo ago Optimization Is Not All You Need arXiv:2607.11977v1 Announce Type: cross Abstract: In 2019, OpenAI released two million GPT-2 outputs-ungrammatical, half broken-to aid the detection of machine-generated text. The alignment that produced their more fluent successors is usually regarded as an engineering… 4 arXiv — NLP / Computation & Language research 1mo ago Growing a Tail: Increasing Output Diversity in Large Language Models arXiv:2411.02989v2 Announce Type: replace Abstract: How diverse are the outputs of large language models when diversity is desired? We examine the diversity of responses of several language models to questions with multiple possible answers, comparing them with human responses.… 4 llama.cpp releases dev-tools 1mo ago b10015 opencl: do not use clCreateBufferWithProperties when targeting CL 2.x ( #25673 ) macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu… 6 r/LocalLLaMA community 1mo ago [audio.cpp] 10 hours of audio generated in 3 minutes on RTX 5090 (demo included)! C++/GGML based Supertonic 3, MOSS-TTS, IndexTTS2, and Irodori-TTS released audio.cpp again. Hopefully you are not sick of it yet :) Release 0.3 adds five new models: Supertonic 3, MOSS-TTS-Local, MOSS-TTS-Nano, IndexTTS2, and Irodori-TTS. The highlight is Supertonic 3. It can hit 200 ×+ real time on CUDA (RTX 5090), 6×+ on CPU, and around 47 ms TTFT in… 37 llama.cpp releases dev-tools 1mo ago b10012 hexagon: fix hmx-queue signal enum-narrowing problem ( #25677 ) macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan)… 15 llama.cpp releases dev-tools 1mo ago b10011 server : refactor prompt cache state ownership ( #25649 ) server : clear checkpoints upon prompt clear server : move the prompt state data to the server_prompt_cache Assisted-by: pi:llama.cpp/Qwen3.6-27B server : handle batched slot being cleared macOS/iOS: macOS Apple Silicon… 38 llama.cpp releases dev-tools 1mo ago b10010 server: add --cors-* options ( #25655 ) server: add --cors-* options add special "localhost" value add tests fix test add link to PR macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64… 33 r/LocalLLaMA community 1mo ago MTP decoding patched for pre-Ampere GPUs (Kepler/Maxwell/Pascal/Turing) The current implementation of multi token prediction (MTP) in llama cpp could trigger BF16 compute selection on GPUs that don't support BF16, causing cuBLAS crashes on older architectures (tested on Kepler). I patched llama.cpp's CUDA backend to add a robust capability check: -… 37 llama.cpp releases dev-tools 1mo ago b10007 opencl: fix a dp4a bug for devices where cl_khr_integer_dot_product is unavailable ( #25639 ) opencl: do not fail backend init on devices without cl_khr_integer_dot_product opencl: do not call dp4 kernels when dp is unavailable Co-authored-by: Li He [email protected]… 10 llama.cpp releases dev-tools 1mo ago b10005 DeepseekV4: fix seq_rm ( #25588 ) DeepseekV4: fix seq_rm implement proper seq_cp create actual update context macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64… 25 NVIDIA Developer Blog official-blog 1mo ago Lessons From the Leaderboard: What 5,000+ Kagglers Taught Us About Improving AI Reasoning The NVIDIA Nemotron Model Reasoning Challenge invited the Kaggle community to explore a focused question: What techniques can improve reasoning accuracy when... 32 llama.cpp releases dev-tools 1mo ago b10004 vulkan/cpu: Support f16 as SET_ROWS src. ( #25432 ) vulkan/cpu: Support f16 as SET_ROWS src. This adds full support for f16 SET_ROWS (equivalent to f32) to vulkan and CPU backends, and adds more backend tests. Set DenormPreserve 16 when supported, to try to fix failures on Intel… 6 r/LocalLLaMA community 1mo ago Bonsai 27B: 1-bit dense LLM running locally in your browser using custom WebGPU kernels Very impressive release by the PrismML team. 1-bit quantization shrinks it from 54GB to just 3.8GB (-93%), while retaining 90% of its intelligence. - Collection on Hugging Face: https://huggingface.co/collections/prism-ml/bonsai-27b - Demo link:… 10 llama.cpp releases dev-tools 1mo ago b10003 tokenize : align usage by using common args ( #25516 ) Migrate the tokenize tool to common_params_parse, replacing its hand-rolled argv parsing, Windows UTF-8 handling and file reading with the shared common helpers. Expose the model-sourcing flags (-m, -mu, -dr, -hf, -hff,… 16 r/MachineLearning community 1mo ago [P] RL-training Qwen3.6 to RL-train tool using AI models [P] 👋 Training my first RL model last year was super fun, now I've RL-trained a model that RL-trains other models... wild times! The agent gets a task, writes the full training job (environment, reward, dataset, hyperparameters), and submits it to real GPUs. When the model it… 10 NVIDIA Developer Blog official-blog 1mo ago How to Run an Autoresearch Workflow with RL Agent Skills and NVIDIA NeMo Coding AI agents are becoming practical operators for long-running machine learning (ML) workflows. They can inspect repositories, set up runtimes, resolve... 17 NVIDIA Developer Blog official-blog 1mo ago Post-Train NVIDIA Cosmos 3 in One Day Using Agent Skills What if autonomous coding AI agents could push your vision reasoning models above 90% accuracy with almost no manual effort? When adapting vision reasoning... 37 llama.cpp releases dev-tools 1mo ago b10002 ggml : add a set of functions for checking contiguity of inner tensor dimensions ( #25650 ) Co-authored-by: Stanisław Szymczyk [email protected] macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework… 27 r/LocalLLaMA community 1mo ago How does MTP actually improve performance? Haven't understood the verification process Hi there, does anyone know how MTP works? I was trying to wrap my head around it, but things aren't adding up. For example, the input string I give it is "look up, " and I want it to output "the sky is blue", except the MTP heads miss the mark and tell me "the sky is green". How… 20 Page 10 of 10 · 500 articles ← Newer
