r/LocalLLaMA · · 1 min read

QAT variant of Gemma4 26B A4B is not working well for me

Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.

QAT variant of Gemma4 26B A4B is not working well for me

I am using llama.cpp version b9549 with this arguments as recommended:

llama-server --temp 1.0 --top-p 0.95 --top-k 64 -hf ...

Here is what I got on chessboard svg test
https://www.reddit.com/r/LocalLLaMA/comments/1t53dhp/quality_comparison_between_qwen_36_27b/

google/gemma-4-26B-A4B-it-qat-q4_0-gguf:IT

google/gemma-4-26B-A4B-it-qat-q4_0-gguf:IT

unsloth/gemma-4-26B-A4B-it-qat-GGUF:Q4_K_XL

unsloth/gemma-4-26B-A4B-it-qat-GGUF:Q4_K_XL

For comparison here is the old gemma4 with the same arguments
unsloth/gemma-4-26B-A4B-it-GGUF:Q4_K_XL

unsloth/gemma-4-26B-A4B-it-GGUF:Q4_K_XL

As you can see old A4B got everything right. I ran it multiple times, it's not perfect, sometimes it swaps color pattern, but at least pieces are rock solid compared to QAT version.

Did anyone try it, do you see the same results?

submitted by /u/pftbest
[link] [comments]

Discussion (0)

Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

Sign in →

No comments yet. Sign in and be the first to say something.

More from r/LocalLLaMA