**Your $80 Tesla P100 has been doing silently noisy math in llama.cpp for years. Three lines fix it, for free.**

## TLDR;

Shipped — in turboquant v0.3.0, downloadable now. https://github.com/TheTom/llama-cpp-turboquant/releases/tag/tqp-v0.3.0

llama.cpp's CUDA code has a flag that means "this GPU is fast at fp16, so do the math in fp16."
The GTX 10-series and P40's (sm_61) were exempted from it long ago. The P100 (sm_60) was not,
ironically, because it's the one Pascal card with fast fp16 hardware. Nvidia put fast FP16
silicon on the P100, so it makes total sense they'd want to tap into that extra performance.
What they didn't check, apparently, was the price.

PR Status: TheTom (merged) https://github.com/TheTom/llama-cpp-turboquant/pull/212
spiritbuun (open) https://github.com/spiritbuun/buun-llama-cpp/pull/80
GGML: Strict Policy on AI-assisted code contributions. I'll try and get around
to hand-writing an issue for them. Highly suggest one of the forks above as an alternative.

The patch is 3 lines.

## Body

A few days ago I was benchmarking buun's new KV-cache codec on my quad-P100 box and comparing notes with the numbers buun was getting on his 3090 — the same model kept showing systematically different quality floors between our machines. I thought everything was the same? Normally, I'd probably just think there's way too many variables between all this code to chalk it up to any one thing… I decided it was worth chasing. It was. And it led me to a serious bug that's been sitting in llama.cpp for years.

So I measured it. Against fp32-reference logits (KL divergence over the full distribution, Qwen3.6-27B, wikitext-2):

Headline:

- **Median KLD: 0.0023 → 0.000001** (~2300× tighter)
- **Top-token agreement: 96.5% → 99.9%** — stock, about 1 in 29 of the model's next-token
picks were different from what the math says they should be

**What's the extra math cost me in performance?** I benchmarked prefill and decode at 8k depth on
three model classes (27B hybrid, 4B dense, 36B MoE). The prefill was identical within noise on all
three, decode actually ~1.4% *faster* patched. The "fast" path was buying nothing: real
workloads on P100 are bound by GEMM and memory bandwidth, not the fp16 vector path. The
patch is 3 lines, extending the exact same exemption sm_61 already had.

**Before everyone freaks out and assumes their 4090 is broken** this is *measured on sm_60 only*.
Your GTX 1080/P40 was always fine (already exempt). Volta and newer are untouched by this patch and
run different kernels entirely — whether other arches have their own unmeasured precision
stories is a separate research question I'm still digging into. Don't read this as "all
GPUs are broken"; read it as "one specific GPU was, and now it isn't."

Post Production Edit - TheTom: "Verified on my side before merging: the three gates are the only 600-vs-610 distinction anywhere in the CUDA tree, so the carved sm_60 path is preprocessor-identical to the long-proven sm_61 path, and a Blackwell build showed bit-identical PPL with decode unchanged, confirming zero effect on other arches."

**Why you might care in 2026:** P100s are going for ~$80 shipped right now while the DRAM
crisis sends everything else to the moon. 16GB of HBM2 at 732 GB/s. The market priced the P40 at ~$300 partly because it "runs better." Some of that gap was this bug.

- Full technical writeup with methodology and receipts: https://gist.github.com/apollo-mg/9218d50a209d70a85f033bf182657818

Found and isolated by running Fable 5 through my custom P/ReAct/R agent loop.
It wrote the scripts, the hardware provided the receipts. Same workflow
as my KV-checkpoint sidecar patch (https://www.reddit.com/r/LocalLLaMA/s/VTIwEFpYgc)
that got merged into turboquant last week.