Qwen vs Gemma
Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.
Hi! Been doing some local LLM stuff, and I can't help but notice: despite vastly-superior benchmark scores, Qwen 3.6 35a3B feels... substantially less intelligent than Gemma 4 26a4B (QAT). In terms of prompt adherence, output coherence, and just general "sanity", Gemma seems head-and-shoulders ahead. Arena.ai rankings generally agree: Gemma 4 26a4 ranks like 7 ELO points below Qwen 3.6 Plus, a larger proprietary model.
I'm running both at Q4, so maybe what I'm seeing is just the QAT... working? But, still, I'm a bit surprised by how big the gap feels.
Has anyone else seem similar? Is there some "trick" to get Qwen to... act smarter? Curious to hear what your experiences have been like.
[link] [comments]
More from r/LocalLLaMA
-
The AI race: China's open-weight models trail US closed models by just 7 months
Oct 14
-
Anthropic Executive Says Open-Source AI Models Are Higher Risk
Oct 14
-
ComfyUI v0.40.0 released
Oct 14
-
So this is how they do it --> RTX 5090, once a consumer gaming card, is now on smugglers’ menu alongside cigarettes
Oct 14
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.