club-3090 adds experimental FP8 support for Qwen3.6-27B!
Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.
It’s finally here! Something many of us running dual RTX 3090 rigs have been anticipating. club-3090 has rolled out experimental support for Qwen3.6-27B with FP8 quantization.
The official Qwen/Qwen3.6-27B-FP8 model performs virtually identically to the original unquantized BF16.
https://github.com/noonghunna/club-3090/blob/master/models/qwen3.6-27b/vllm/compose/dual/fp8/mtp.yml
[link] [comments]
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.