MiMo v2.5 is underrated. Feels like the tokens are pouring out of the screen in OpenCode.

When I recently built my inference server, I expected to deploy DeepSeek v4 flash, but that doesn't look like it's going to be fast for a long time, if ever. There is a massive gap, as we all know, in competent models between 30b and 400b. I was very surprised to find that this is the best model by far that falls within this gap. MiMo v2.5 the only local model I have seen that's faster than fast cloud providers. It is actually worth it to build a big server just to run this thing.

Running 192GB 4090 VRAM, I tested Bartowski IQ4_XS, IQ4_NL, Unsloth UD-Q4_K_S, and gghfez "unfused" IQ4_XS in ik_llama.

• The Bartowski IQ4_NL seems like the best quality and speed so far.
• I ran the unfused IQ4_XS in --split-mode graph. It was fast but noticably dumber and more prone to looping.
• UD-Q4_K_S seems equivalent to IQ4_NL in quality, but maybe ~1000pp/10tg slower than NL.
• The image above was obtained by throwing random .md files at the model. The numbers get higher up to about 20kb worth of files.

Looping

• Looping is definitely an issue. It will repeat chunks of thinking forever until the max tokens are reached.
• My current settings are --temp 1.0 --top-p 0.95 --repeat-penalty 1.2 --repeat-last-n 128. This seems to work very well. I tested by repeatedly running a request in OpenCode that caused a massive loop with different settings until it stopped.
• I had some settings that appeared to mitigate looping at first, but actually made the model incapable of using tools or thinking clearly: --repeat-penalty 1.5 --presence-penalty 0.4 --frequency-penalty 0.2. Not recommended. I'm including these as a warning. Such penalties can drive the model insane. It's very sensitive.

Optimizations

• MTP doesn't work in llama.cpp as far as I know. There's an open Github issue.
• --split-mode tensor doesn't work in llama.cpp. There's an open Github issue for this as well.
• --split-mode graph does work in ik_llama, but the only model I found with ik_llama-compatible "unfused" tensors isn't great.
• As if it really needs to be faster than it already is.

Multimodal

• Vision - Now that I think about it, I'm actually not sure if this working. Didn't investigate much. I assume not in llama.cpp.
• This model actually does ASR! Not working yet in llama.cpp, but there is an open issue. Literally you will be able to talk directly to it and have your tone, inflection, background noises understood by the model if this ever gets working. I'm currently using Qwen3 ASR so it would be a huge upgrade.