r/LocalLLaMA · · 1 min read

Updates on North Mini Code: 4 bit quant + Ollama + OpenRouter

Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.

Updates on North Mini Code: 4 bit quant + Ollama + OpenRouter

Hey!

We heard the feedback on making the model more portable and accessible. So in light of that we have 2 updates to share.

First, you can pull a new 4-bit quant straight from Hugging Face, so it’s now small enough to run on a Mac or whatever local hardware you’ve got. It needs about 20 gigs so if you have that you are good to go.

Second, North Mini Code is now supported on Ollama, and any other local runtimes built atop llama.cpp, and it’s also available via the OpenRouter API. we know a lot of you wanted more access, so hoping this lets more devs build more cool stuff.

The full docs are here. Excited to hear what you guys think :)

submitted by /u/nick_frosst
[link] [comments]

Discussion (0)

Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

Sign in →

No comments yet. Sign in and be the first to say something.

More from r/LocalLLaMA