I benchmarked 21 Qwen3.8 27B variants on 16GB VRAM
Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.
After Qwen3.8 27B came out, I decided to benchmark the models that could fit in my GPU (RTX 5080) on my actual code (C code), the results were not completely unexpected but some quants were definitely underwhelming.
TLDR: Best overall: bartowski/Qwen3.8-27B-IQ4_XS. Best uncensored: huihui-ai/Huihui-Qwen3.8-27B-abliterated-UD-IQ4_XS. For a bit more context: jpetrina/Qwen3.8-27B-IQ4_XS-pure or uncensored: Bucoid/Qwen3.8-27B-Uncensored-IQ4_XS_4BPW
(sorted by Mean KLD)
| Model | Mean KLD | Same top p | GGUF size |
|---|---|---|---|
| sdkyuan/qwen38-27b-qat-q2_0 | 0.893177 ± 0.006948 | 85.727 ± 0.110 % | 8.2GiB |
| ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-IQ2_XS.gguf | 0.767174 ± 0.006291 | 86.166 ± 0.108 % | 7.8GiB |
| ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-IQ2_S | 0.512614 ± 0.004909 | 88.802 ± 0.099 % | 8.6GiB |
| empero-ai/Qwen3.8-27B-Ridge-3.7bpw | 0.475767 ± 0.004483 | 89.612 ± 0.096 % | 11.7GiB |
| ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-IQ3_XXS | 0.379222 ± 0.003992 | 90.270 ± 0.093 % | 9.4GiB |
| unsloth/Qwen3.8-27B-UD-Q2_K_XL (UD2) | 0.350861 ± 0.003745 | 90.626 ± 0.091 % | 9.9GiB |
| unsloth/Qwen3.8-27B-UD-IQ3_XXS (UD2) | 0.268594 ± 0.002971 | 91.951 ± 0.085 % | 11.1GiB |
| DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NM-DAU-NEO-MAX-NEO-MTP-IQ3_M | 0.251270 ± 0.002702 | 92.315 ± 0.083 % | 13.5GiB |
| esatapedico/Qwen3.8-27B-NVFP4-MTP-LOW | 0.220796 ± 0.002631 | 92.339 ± 0.083 % | 14.5GiB |
| mudler/Qwen3.8-27B-APEX-I-Mini | 0.190209 ± 0.002354 | 93.012 ± 0.080 % | 13.0GiB |
| jrell/Qwen3.8-27B-i1-IQ4_XS-GGUF-Smaller | 0.194459 ± 0.002242 | 93.049 ± 0.080 % | 12.6GiB |
| orcarouter/Qwen3.8-27B-Uncensored-Q3_K_L | 0.192312 ± 0.002294 | 92.726 ± 0.081 % | 13.6GiB |
| unsloth/Qwen3.8-27B-UD-Q3_K_XL (UD2) | 0.147186 ± 0.001809 | 93.734 ± 0.076 % | 12.5GiB |
| unsloth/Qwen3.8-27B-UD-Q3_K_XL (UD3) | 0.142647 ± 0.001860 | 93.789 ± 0.076 % | 12.2GiB |
| Bucoid/Qwen3.8-27B-Uncensored-IQ4_XS_4BPW | 0.091447 ± 0.001261 | 94.774 ± 0.070 % | 13.0GiB |
| huihui-ai/Huihui-Qwen3.8-27B-abliterated-UD-IQ4_XS | 0.082871 ± 0.001205 | 94.981 ± 0.068 % | 13.4GiB |
| unsloth/Qwen3.8-27B-UD-IQ4_XS (UD3) | 0.075626 ± 0.001097 | 95.258 ± 0.067 % | 13.3GiB |
| jpetrina/Qwen3.8-27B-IQ4_XS-pure | 0.061984 ± 0.000917 | 95.551 ± 0.065 % | 13.5GiB |
| bartowski/Qwen3.8-27B-IQ4_XS | 0.056482 ± 0.000856 | 95.835 ± 0.063 % | 14.5GiB |
| unsloth/Qwen3.8-27B-UD-Q4_K_XL (UD3) (can't fit) | 0.029844 ± 0.000476 | 96.921 ± 0.054 % | 16.4GiB |
| unsloth/Qwen3.8-27B-UD-Q4_K_XL (UD2) (can't fit) | 0.028026 ± 0.000432 | 96.988 ± 0.054 % | 16.7GiB |
Hope this helps other VRAM starved people like me :)
[link] [comments]
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.