Faster than Light in Air: 8-22 tg/s Qwen3.8-Flash-Next (Q4/Q4ish) on a 32GB M4 MacBook Air

I believe I may currently hold the record for memory constrained inference for Qwen3.8–Flash-Next on Apple Silicon — needing only about 21GB of allocations.

Introducing Cherenkov, an inference engine for Apple Silicon combining predictive expert streaming with optional mixed-precision execution. It works by keeping a bounded working set of experts in unified memory, rather than loading the entire model. A one-layer lookahead predicts which experts will be needed next and initiates SSD reads. If there isn’t enough time to load the full target expert, Cherenkov optionally falls back to just in time loading of smaller Q3/Q2 quantizations.