News / #gpu Tag Gpu 500 articles archived under #gpu · RSS Sign in to follow arXiv — NLP / Computation & Language research 21d ago Confidently Deceptive: How Confidence Amplifies the Risk of LLM Deception arXiv:2607.20444v1 Announce Type: new Abstract: Large language models (LLMs) can produce deceptive responses: outputs that mislead users in service of a contextually or experimentally induced goal. Yet it remains unclear how confidently models deceive and whether higher… 24 arXiv — NLP / Computation & Language research 21d ago Response drift across frontier large language models arXiv:2607.20454v1 Announce Type: new Abstract: All frontier large language models (LLMs) exhibit response drift -- producing outputs that deviate from expert-validated references -- yet the magnitude and structure of this drift remain uncharacterised by systematic human… 31 arXiv — NLP / Computation & Language research 21d ago Learning to Detect UI Principle Violations via Reinforcement Learning arXiv:2607.20690v1 Announce Type: new Abstract: Small language models and coding agents increasingly generate web front-end code, yet their outputs are typically evaluated primarily for functional correctness. A generated interface may compile, render, and pass unit tests while… 15 arXiv — NLP / Computation & Language research 21d ago Beyond Liars' Bench: The Impact of Lie Typology, Depth, and Sparsity on Deception Detection in LLMs arXiv:2607.20479v1 Announce Type: cross Abstract: Training probes to detect deceptive outputs from large language models is still an open problem. Recent work has demonstrated that detection probes fail especially in out-of-domain scenarios -- training on one type of lie does… 35 arXiv — NLP / Computation & Language research 21d ago NVIDIA-labs OO Agents: Native Python Object-Oriented Agents arXiv:2607.20709v1 Announce Type: cross Abstract: Traditional agent development is split across prompt templates, tool schemas, callback code, and workflow graphs. We present NVIDIA Object-Oriented Agents (NOOA), a model-agnostic Python framework for building reliable AI agents.… 32 llama.cpp releases dev-tools 21d ago b10099 CUDA: Improve NVFP4 W4A4 activation quantization ( #25730 ) Squash history before conflict-resolution during rebase on master WIP commit Add 32-byte loads, restore per-block amax Use nvfp4x4 intrinsic when available Fuse per-channel amax and quantization kernels Do pointer… 6 TechCrunch — AI news-outlet 21d ago AMD takes on Nvidia with its Helios AI rack scale system AMD is challenging its chipmaker rival with a new rack-scale system that will start shipping to customers later this year. 32 LangChain releases dev-tools 21d ago langchain-anthropic==1.5.1 Changes since langchain-anthropic==1.5.0 release(anthropic): 1.5.1 ( #39043 ) feat(anthropic,fireworks,openai): support langsmith gateway through env var ( #38742 ) fix(anthropic): enable structured output for Claude Opus 4.8 ( #39021 ) 27 llama.cpp releases dev-tools 21d ago b10098 hexagon: activation ops update ( #25974 ) hex-geglu: optimized all-in-one geglu microkernel hex-geglu: enable non-contiguous src and strided DMA hex-act: enable non-contiguous srs and strided DMA for rest of ACT ops hex-act: generalize GLU per-thread functions via… 28 r/LocalLLaMA community 21d ago Laguna-S-2.1 "thinking forever" loops seem to be a quantization artifact If you're running Laguna S 2.1 on llama.cpp and hitting thinking loops because it won't close its </think> tags, you might want to look at your quant before you spend too much time tweaking settings. I spent a day debugging this, and here is what finally gave me clean outputs:… 24 r/LocalLLaMA community 21d ago Deepseek V4 Flash ~105 t/s on two Nvidia 4090d 48G (ada) in vLLM TLDR: I (with the help of AI) re-implemented every Blackwell-only kernel (DeepGEMM, FlashInfer sparse-MLA, block-scaled FP8) in Triton, because they simply don't exist for sm89. The performance is 2-3x more for parallel agentic workflows. Benchmark llama-server vs vLLM I was… 27 llama.cpp releases dev-tools 21d ago b10094 common: infer the speculative type from the draft repo sidecars ( #25989 ) With -hfd pointing to a repo that ships mtp-/dflash-/eagle3- sidecars and no --spec-type given, the draft resolved to a full model while the sidecar was the intended draft. When the speculative types are… 19 llama.cpp releases dev-tools 21d ago b10093 Fix DeepSeek4 crafted template ( #25414 ) chat: fix DS4 template to explicitly follow reference behavior Support DeepSeekv4 flag ( drop_reasoning ). fix: hook DS3.2 parser for DS4 as well fix: add tool result reordering fix: post-merge Website: https://llama.app macOS/iOS: macOS… 20 NVIDIA Developer Blog official-blog 21d ago Debugging Ray Tracing Applications Using NVIDIA OptiX Toolkit NVIDIA OptiX ray tracing engine is an application framework for achieving optimal ray tracing performance on the GPU. Applications using OptiX can fail in ways... 37 NVIDIA Developer Blog official-blog 21d ago Start Customizing NVIDIA Nemotron 3 Nano with Prime Intellect Lab in Minutes Customization is what enables developers to take a general model and tailor it to use cases, domains, languages, and more. However, customization comes with a... 35 TechCrunch — AI news-outlet 21d ago AI chip startup Etched defies skeptics, hits $10.3B valuation from big-name investors Etched, founded by three Harvard dropouts, has created new chips and memory components that speed up inference on any AI model -- no GPUs required, it says. 14 TechCrunch — AI news-outlet 21d ago Nvidia is sending GPUs to the moon If there's a place in the universe without GPUs, Nvidia is sending them there. 6 llama.cpp releases dev-tools 22d ago b10092 ggml: enable PowerPC backend variants on AIX ( #25983 ) ggml: enable PowerPC backend variants on AIX Allow the PowerPC CPU backend variants to be built on AIX by extending the platform check in the CMake configuration. This reuses the existing PowerPC backend implementations… 5 r/LocalLLaMA community 22d ago Model "distillation" accusations are getting way overblown at this point Every time a strong open model drops, the same cycle plays out: ai bro's claims it's "just distilled from GPT4/Claude/whatever," case closed, move on. I think this take doesn't hold up as well as people assume. A few points worth separating out: Training on outputs isn't the… 25 arXiv — Machine Learning research 22d ago Nonlinear Bias-Compensated Adaptive Filter and Its Application for Time-Series Prediction arXiv:2607.19902v1 Announce Type: new Abstract: Most existing nonlinear adaptive filtering algorithms only account for output noise, neglecting the fact that input noise is also prevalent in practice. Although the recently proposed bias-compensated kernel least mean square… 13 arXiv — Machine Learning research 22d ago PhaseAware: Interpretable Human-in-the-Loop Rehabilitation Scoring with Boundary Monitoring arXiv:2607.20237v1 Announce Type: new Abstract: Rehabilitation scoring systems are most useful when their outputs can be reviewed and interpreted within clinical workflows. This study presents PhaseAware, a compact framework for continuous rehabilitation quality assessment that… 13 arXiv — Machine Learning research 22d ago Benchmarking Confidential GPU Inference on NVIDIA H100 under Intel TDX arXiv:2607.19353v1 Announce Type: cross Abstract: Confidential computing is becoming a practical deployment requirement for AI inference workloads that process sensitive inputs or protect proprietary model assets. However, the performance cost of enabling confidential execution… 26 arXiv — NLP / Computation & Language research 22d ago D2VBench: Benchmarking Large Language Models with Value Dilemmas in Daily Scenarios arXiv:2607.19834v1 Announce Type: new Abstract: With the wide application of large language models (LLMs) in real-world scenarios, the value implication of their outputs is crucial. However, existing evaluation benchmarks suffer from insufficient coverage of value dilemmas in… 34 arXiv — NLP / Computation & Language research 22d ago Sound Probabilistic Safety Bounds for Large Language Models arXiv:2607.20286v1 Announce Type: new Abstract: We propose a novel framework for computing rigorous bounds on the probability that a large language model (LLM) generates harmful output to a given prompt. We study a new application of the Clopper-Pearson confidence intervals to… 16 arXiv — NLP / Computation & Language research 22d ago BaseRT: Advancing Best-in-Class LLM Inference with Apple M5 Neural Accelerators arXiv:2607.19438v1 Announce Type: cross Abstract: Apple's M5 generation introduces a redesigned GPU architecture in which every core carries a dedicated Neural Accelerator: on-die matrix units exposed through the Metal~4 tensor API. We show that BaseRT, our native Metal… 8 arXiv — NLP / Computation & Language research 22d ago Reading and Steering Representations of Materials-Science Mechanisms in an Open-Weight Language Model arXiv:2607.20058v1 Announce Type: cross Abstract: Large language models can answer scientific questions, yet a correct output does not reveal whether the model represents or uses the governing physics. Here we show that materials science mechanism information in the open-weight… 27 arXiv — NLP / Computation & Language research 22d ago ENTRAP-VL: A Taxonomic Probe for Dual Contextual Entrainment in Vision-Language Models arXiv:2607.20092v1 Announce Type: cross Abstract: Contextual entrainment is the tendency of a model to let auxiliary context in its input pull its output, independently of whether that context is relevant, true, or even meaningful. Recently, it has been identified and given a… 23 arXiv — NLP / Computation & Language research 22d ago Self-Preference Bias in Rubric-Based Evaluation of Large Language Models arXiv:2604.06996v2 Announce Type: replace Abstract: LLM-as-a-judge has become the de facto approach for evaluating LLM outputs. However, judges are known to exhibit self-preference bias (SPB): they tend to favor outputs produced by themselves or by models from their own family.… 13 r/LocalLLaMA community 22d ago FYI You dont need expensive networking for multi-node gpu. 30t/s laguna Q2_K_XL (39.7GB) on 2x4060+1x4060 using a $20 usb->ethernet. Turns out a regular ethernet cable between 2 nodes can run laguna UD-Q2_K_XL (39.7GB) using a direct point to point network. Interestingly on `nvidia-smi dmon -s pucvmet -d 2`, the inter/intra gpu traffic is not really capped in this setup - Uses ~30-70MB/s at peak # gpu pwr… 26 llama.cpp releases dev-tools 22d ago b10091 ci : fix SYCL package shared library lookup ( #25987 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu… 5 llama.cpp releases dev-tools 22d ago b10090 webgpu : add CONV_2D_DW (depthwise conv2d) kernel ( #25847 ) webgpu : add CONV_2D_DW (depthwise conv2d) kernel Implement GGML_OP_CONV_2D_DW for the WebGPU backend, ported from the Vulkan backend's conv2d_dw.comp. Assisted-by: Claude Opus-4.8 Remove unnecessary comments in webgpu… 37 llama.cpp releases dev-tools 22d ago b10089 cuda: GET_ROWS quants ( #25962 ) cuda: add k-quant support to GET_ROWS Device-side embedding lookups require GET_ROWS to handle the k-quants used by common GGUF recipes (Q4_K_M stores token_embd as q6_K). Without it the backend rejects the op and the scheduler falls back to the… 31 r/LocalLLaMA community 22d ago 16x AMD MI50 32GB: GLM-5.2 Q4 at 12.2 tok/s with llama.cpp RPC GLM-5.2 UD-Q4_K_XL GGUF @ 12.2 tok/s output // 30.9 tok/s input on a real 10.7k-token document using llama.cpp RPC - At 10.7k context: 10.2 tok/s output with coherent long-form generation Two parallel requests: 14.5 tok/s aggregate Context: 2x 16,384-token slots Model size: 436… 37 NVIDIA Developer Blog official-blog 22d ago Make Long-Running NVIDIA TensorRT Engine Builds Observable and Cancelable in Python or C++ A TensorRT engine build can take seconds to many minutes. Large strongly typed models, deep tactic search, and a cold timing cache on a brand-new GPU SKU can... 14 r/LocalLLaMA community 23d ago Llama.cpp just added support for Laguna XS.2 & M.1 https://github.com/ggml-org/llama.cpp/releases/tag/b10087   submitted by   /u/LaurentPayot [link]   [comments] 29 llama.cpp releases dev-tools 23d ago b10087 Add support for Laguna XS.2 & M.1 ( #25165 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64… 7 llama.cpp releases dev-tools 23d ago b10085 mtmd : use align_corners for qwen3vl vision position embedding interpolation ( #25781 ) The Qwen3-VL learned position embedding is interpolated to the runtime patch grid with the default bilinear+antialias (align_corners=False) sampling, while the transformers reference uses… 32 llama.cpp releases dev-tools 23d ago b10084 hexagon: check tensor type when reusing descriptors ( #25968 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU)… 38 llama.cpp releases dev-tools 23d ago b10083 cuda: add sqrt_softplus in topk-moe for dsv4 ( #25896 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu… 33 r/LocalLLaMA community 23d ago Force <thinking> in Laguna-S-2.1 Those who have tested the new Laguna model might have noticed how reluctant it is to think through medium-hard questions, and it does impact the output quality. It is great that the model does not "Qwen over" questions like "Hi, who are you", but it definitely should think more.… 32 r/MachineLearning community 23d ago SkewAdam: A tiered optimizer that cuts MoE state memory by 97% (fits a 6.7B MoE on a 40GB GPU) [R] Paper: https://arxiv.org/abs/2607.19058 Code (GitHub): https://github.com/nuemaan/skewadam Hi everyone, I just published a preprint on a new optimizer designed to tackle the massive VRAM bottleneck in Mixture-of-Experts (MoE) training. If you've trained MoEs, you know that… 24 llama.cpp releases dev-tools 23d ago b10082 kleidiai : warn once when a weight type has no KleidiAI kernel ( #25701 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu… 21 r/LocalLLaMA community 23d ago VRAM disk cache of MoE makes 340 pp/s 9.6 tg/s for Kimi 2.7 on a single dgx spark this strategy effectively uses vram as cache over disk to keep MoE experts on cuda compute path in llama.cpp. numbers first. detailed explanation down below. Numbers on dgx spark Kimi-K2.7-Code.i1-IQ_S.gguf 204GB 1T.A32B https://huggingface.co/mradermacher/Kimi-K2.7-Code-i1-GGUF… 20 llama.cpp releases dev-tools 23d ago b10081 common: resolve draft repo to its requested sidecar ( #25955 ) With -hfd pointing to a repo shipping speculative sidecars, the draft resolved to the main model of that repo, since find_best_model() excludes sidecar files, and the explicit draft plan suppressed the sidecar… 26 llama.cpp releases dev-tools 23d ago b10080 server: return 400 instead of 500 on validation error with X-Conversation-Id ( #25760 ) server: return 400 instead of 500 on validation error with X-Conversation-Id set_req() attaches the spipe as soon as the header is present, before the request body is parsed. When params… 5 Hugging Face Daily Papers research 23d ago ABot-World-0: Infinite Interactive World Rollout on a Single Desktop GPU Abstract We present ABot-World-0, an action-conditioned video world model for real-time, long-horizon closed-loop interaction, supported by a multi-source data infrastructure spanning AAA games, simulation engines, and internet videos to learn controllable world dynamics.… 19 Hugging Face Daily Papers research 23d ago HPD-Parsing: Hierarchical Parallel Document Parsing Abstract Efficient teamwork typically combines global coordination with parallel execution, a principle not yet fully reflected in unified Vision-Language Model (VLM)-based document parsers. Existing unified parsers process an entire page jointly but generate its output through… 26 arXiv — Machine Learning research 23d ago Beyond Output-Space Calibration: Spectral Evidence Bundling for Selective Reliability Estimation in Time-Series Classification arXiv:2607.18279v1 Announce Type: new Abstract: Post-hoc calibration for time-series classification usually remaps output scores, but deployment decisions such as trust, abstention, and review depend on whether a confident prediction is supported by the current temporal signal.… 21 arXiv — Machine Learning research 23d ago On the Limits of Support-Preserving Alignment and Bounded Filtering arXiv:2607.18295v1 Announce Type: new Abstract: We study whether alignment schemes that reshape a base model's output distribution, combined with bounded safety filters, can drive the probability of harmful behavior to zero in modern large language models. Recent research… 11 arXiv — Machine Learning research 23d ago Conditioned Direct Feedback Alignment via Activity and Error Geometry arXiv:2607.18574v1 Announce Type: new Abstract: Direct feedback alignment (DFA) trains hidden layers with fixed random projections of the output error, avoiding the transposed-weight backward pass of backpropagation (BP). We study a failure mode of DFA training that is distinct… 9 Page 7 of 10 · 500 articles ← Newer Older →
