24GB M4 Mac - is Qwen 9B only option while system is running?
Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.
I have mac at work that I want to use local model for prototyping and basic prompts that needs to stay on device. What sort of model I can run that I can fit at least 64k context ? Any setups share or guides welcome.
I need to have firefox open with one tab at minium. Problem I have is all the crap that runs on Mac itself by default.
[link] [comments]
More from r/LocalLLaMA
-
Re. what ever happened to Cohere’s Command-A series of models?
May 20
-
Qwen will release another 27B with high probability
May 20
-
I got Qwen3-VL-Embedding-2B working with rkllm on an Orange Pi 5b
May 20
-
Move to backend sampling for MTP draft path by gaugarg-nv · Pull Request #23287 · ggml-org/llama.cpp
May 20
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.