DeepSeek V4 Flash | IQ3_XXS-AS & IQ2_S Bench | mainline b10064 vs fairydreaming | 1xRTX 3090 + 128GB DDR4 | 250PP/11TG on 50K CTX

Hey all!

Wanted to see how DeepSeek V4 Flash GGUFs in two different quants perform on my hardware and share the results.

Tested two quants on the fairydreaming/llama.cpp dsv4 fork. As a bonus, I also ran the same model (IQ3_XXS-AS) on mainline llama.cpp b10064 just to check. TLDR: turns out the fork is pointless now!

Model

DeepSeek V4 Flash made by bullerwins:

• DeepSeek-V4-Flash.IQ3_XXS-AS.gguf
• DeepSeek-V4-Flash.IQ2_S.gguf

Hardware

Component	Spec
CPU	AMD Ryzen 5950X (16C/32T, 14 threads used)
RAM	128 GB DDR4 3600 MHz CL16 (dual channel)
GPU	RTX 3090 24 GB
OS	Win 11 Pro

Build Details

fairydreaming/llama.cpp (dsv4 branch)

-DGGML_CUDA=ON -DGGML_CUDA_FA_ALL_QUANTS=ON -DCMAKE_CUDA_ARCHITECTURES=native -DGGML_CCACHE=OFF -DGGML_NATIVE=ON -DCMAKE_BUILD_TYPE=Release -DLLAMA_OPENSSL=ON -DCMAKE_ASM_COMPILER=<nasm.exe>

Mainline llama.cpp (build b10064)

-DGGML_CUDA=ON -DGGML_NATIVE=ON -DGGML_CUDA_FA_ALL_QUANTS=ON -DCMAKE_CUDA_ARCHITECTURES=native -DCMAKE_BUILD_TYPE=Release -DCMAKE_ASM_COMPILER=<nasm.exe>

Launch Parameters (identical for all runs and models)

llama-server.exe ^ -m "model.gguf" ^ --chat-template-file "deepseek-ai-DeepSeek-V4.jinja" ^ --ctx-size 196608 ^ --jinja --metrics --perf ^ -np 1 ^ -ub 4096 -b 4096 ^ --no-kv-unified ^ --no-mmap ^ --flash-attn on ^ --cache-type-k q8_0 ^ --cache-type-v q8_0 ^ --temp 1.0 --top_k 40 --top_p 1.0 ^ --min-p 0.00 --repeat-penalty 1.0 --presence-penalty 0.0 ^ --threads 14 ^

---

Part 1: IQ3_XXS-AS vs IQ2_S on fairydreaming fork

Cold Start (first request, no cache)

Metric	IQ3_XXS-AS	IQ2_S
Prompt tokens	11,630	11,650
Prompt eval time	44.28 s	41.48 s
Prompt speed	262.66 tok/s	280.84 tok/s
Prompt ms/tok	3.81 ms	3.56 ms
Generation tokens	473	121
Generation time	41.42 s	8.97 s
Generation speed	11.42 tok/s	13.50 tok/s
Generation ms/tok	87.58 ms	74.09 ms
Total time	85.70 s	50.45 s
Total tokens	12,103	11,771
Graphs reused	464	120

Warm Request (cache hit, context reuse)

Metric	IQ3_XXS-AS	IQ2_S
Prompt tokens	40,840	51,150
Prompt eval time	161.37 s	201.33 s
Prompt speed	253.08 tok/s	254.06 tok/s
Prompt ms/tok	3.95 ms	3.94 ms
Generation tokens	2,639	1,671
Generation time	244.56 s	129.87 s
Generation speed	10.79 tok/s	12.87 tok/s
Generation ms/tok	92.67 ms	77.72 ms
Total time	405.93 s	331.20 s
Total tokens	43,479	52,821
Graphs reused	3,057	1,762
Slot total tokens	55,104 (no truncation)	64,466 (no truncation)

Quick Summary

Prompt processing: IQ2_S is slightly faster cold (+7%), but virtually identical warm (both ~254 tok/s). Prompt processing converges at larger contexts.

Generation: IQ2_S is consistently ~18-19% faster (13.50 vs 11.42 cold, 12.87 vs 10.79 warm).

My take: IQ3_XXS-AS is the real sweet spot. IQ2_S isn't faster at PP, yet the difference in quality is noticeable; TG isn't all that important.

---

Part 2: fairydreaming fork vs mainline b10064 (IQ3_XXS-AS)

After testing on the fork, I ran the same IQ3_XXS-AS model on mainline llama.cpp build b10064 with identical settings and task. Here's the comparison:

Cold Start

Metric	dsv4 fork	mainline b10064
Prompt speed	262.66 tok/s	227.38 tok/s
Generation speed	11.42 tok/s	11.88 tok/s

Warm Request

Metric	dsv4 fork	mainline b10064
Prompt speed	253.08 tok/s	251.50 tok/s
Generation speed	10.79 tok/s	11.11 tok/s

Quick Summary

Mainline is essentially identical - within 3-4% in both prompt and generation. There's no practical difference.

---

Bottom Line

The fairydreaming dsv4 fork is no longer needed. Mainline llama.cpp (build b10064+) handles DeepSeek V4 Flash just fine with identical performance. If you're building from source today, just use upstream.

Regarding quantization: I think bullerwins' GGUFs are currently the best available for this model and hardware. As for which quant to pick - IQ3_XXS-AS is the real sweet spot. 2-bit variants run at roughly the same token speed. If speed is the same and IQ3 gives you noticeably better quality, the lower quants aren't worth it.

Performance ceiling: The bottleneck here is likely system memory bandwidth and/or the CPU. During inference, the RTX 3090 consumes only ~150–200 W (with a TDP limit of around 375–400 W), so it still has some compute headroom. Maybe it’s worth buying a 5090 and offloading more layers to VRAM? Maybe I can get at least 500 PP and 25 TG?..

---

Anyway, hope this helps someone.