Qwen 3.8 27B at ~3 BPW on an RTX 3060: GSQ vs ByteShape IQ3-XXS 2.88BPW
Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.
| Someone recommended that I try the ByteShape Qwen 3.8 27B IQ3-XXS GGUF after seeing my previous testing of the GSQ quant. So I did. And the result was… surprisingly bad. For context, I'm running:
The two low-bit quants I compared were: ISTA-DASLab / GSQ-RCO-IQ3-XXS
ByteShape IQ3-XXS
On paper, the ByteShape quant looked very interesting. It was smaller, while apparently retaining extremely high similarity to the original BF16 model. It was also being compared in size to significantly higher-BPW quants. So naturally I expected it to at least be competitive with the GSQ version. It wasn't. The actual result is shown AboveISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF was able to generate a 3D voxel diorama in one shot under 55k tokens, and Byteshape's 3.8 27B model, took roughly three shots and still hasn't completed with over 98K tokens spent already. Same with web development not impressive as advertised in Here Any New Model Suggestions for RTX 3060? [link] [comments] |
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.