QAT variant of Gemma4 26B A4B is not working well for me
Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.
| I am using llama.cpp version b9549 with this arguments as recommended: llama-server --temp 1.0 --top-p 0.95 --top-k 64 -hf ... Here is what I got on chessboard svg test google/gemma-4-26B-A4B-it-qat-q4_0-gguf:IT google/gemma-4-26B-A4B-it-qat-q4_0-gguf:IT unsloth/gemma-4-26B-A4B-it-qat-GGUF:Q4_K_XL unsloth/gemma-4-26B-A4B-it-qat-GGUF:Q4_K_XL For comparison here is the old gemma4 with the same arguments unsloth/gemma-4-26B-A4B-it-GGUF:Q4_K_XL As you can see old A4B got everything right. I ran it multiple times, it's not perfect, sometimes it swaps color pattern, but at least pieces are rock solid compared to QAT version. Did anyone try it, do you see the same results? [link] [comments] |
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.