Mobile Offline LLMs: What do you use them for?
Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.
I've spent the last year or so playing around with open source MLX and GGUF models on iPhone hardware.
Given the limitations in memory, GPU/CPU/ANE, and in turn the context window I've been trying to figure out the best use cases for them.
I've also done a lot of testing with Apple Foundation model. It is not that capable, but it is good at understanding instructions for tool calling and actually seems to work well for fast summarization and classification prior to feeding data to more capable models.
I've been able to get models ranging from 0.5B to 8B to work well on the higher end devices.
So far I've been testing it out with building tooling such as web search, url scraping, summarization, research, analyzing local photo/video/documents/etc., and basic on-the-go coding help. With continuous compaction I've been able to hold virtually unlimited exchange chats even though context windows are limited to like 8-16k tokens.
Separate from more professional use cases, It also seems like potentially a good option for private, offline chat about your personal thoughts with models tuned with custom system prompts.
I know many here are focused on trying to run as good of a model as possible on their laptop or desktop hardware but it seems like mobile is an area that isn't really being explored extensively yet. What do you use mobile models for?
[link] [comments]
More from r/LocalLLaMA
-
Anthropic, OpenAI, Google, Meta, Microsoft, and Mistral all signed the EU Code of Practice on Transparency of AI-Generated Content
Aug 12
-
We quantized DeepSeek V4 0731 and benchmarked it against popular quants on 8× RTX 5090
Aug 11
-
366 t/s Qwen3.6 27B NVFP4 on v100s
Aug 11
-
Local Benchmark : Muse Glimmer 30B vs Qwen 3.6 27B vs Gemma4 31B (and many other models and finetunes)
Aug 11
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.