Sharing two GGUF quant sets, both with the same treatment: imatrix quantization, KLD/PPL measured against BF16 reference logits, llama-bench throughput numbers, and all raw benchmark data included in the repos. No vibes-based "quality tested" claims — every number is reproducible from the files in the repo.

1. Hy3 — Tencent's 295B MoE (21B active)

LordNeel/Hy3-GGUF

Base: tencent/Hy3 — 295B MoE, 21B active, 262K context, Apache 2.0. Converted from BF16 with a Hy3-enabled llama.cpp branch, imatrix from a custom calibration corpus.

*8x RTX PRO 6000 Blackwell, layer split, F16 KV cache. Quality: WikiText-2, 128 chunks x 512 ctx vs BF16 no-MTP reference logits.

Picks: Q6_K is effectively lossless if you have ~256 GB for it. Q4_K_M is the sane default (fits 2x 96 GB GPUs). IQ2_M squeezes onto a single 96 GB card or a 128 GB Mac, but the quality drop is real.

Gotchas: use --split-mode layer (tensor split crashed in CUDA decode on this arch), and the MTP/NextN block is excluded on purpose. The imatrix, calibration corpus, and full repro script are in the repo if you want to roll your own quants.

2. Nemotron-Labs-Audex-30B-A3B — NVIDIA's audio-capable 30B hybrid MoE

LordNeel/Nemotron-Labs-Audex-30B-A3B-GGUF

Base: nvidia/Nemotron-Labs-Audex-30B-A3B — 30B Nemotron-H hybrid MoE, ~3B active per token, 1M context, audio understanding + generation. Two tracks in one repo:

• quants/ — text-only GGUFs for plain llama.cpp use
• audio_quants/ — full-vocab audiogen GGUFs + an audio_support/ sidecar (NV-Whisper encoder, causal speech decoder, enhancement VAE, and NVIDIA's HF/vLLM scripts for TTS, audio QA, and speech-to-speech)

Quality measured over three corpora (WikiText-2 / code / GSM8K), throughput on 2x RTX PRO 6000 Max-Q. Text-only track:

BF16 generates at ~177 tok/s on the same box, so the quants run about 2x faster. MXFP4_MOE is experimental but fun: smallest file and 11.5K prompt tok/s vs ~8.5K for the K-quants. Audiogen numbers are close and in the model card, with per-corpus breakdowns and charts.

Before you start using them:

• License is NVIDIA OneWay Noncommercial (inherited from upstream), not for commercial use.
• llama.cpp runs the nemotron_h LM side; the full audio pipeline needs the sidecar + NVIDIA's scripts. There's no all in one audio GGUF runtime yet.
• ~133 of 401 tensors fall back during K-quantization (Nemotron-H MoE/SSM tensor shapes), which is why Q6_K lands near Q8_0 size.

Both repos have charts, checksums, quant plans, and per-quant logs. Questions and quant requests welcome.