Hey r/LocalLLaMA! :D I wanna share a really cool fully OSS thing I've been building that's only possible with local models: truly proactive AI!

All your existing LLM systems waits for a prompt. Truly proactive AI has to read your entire life, every single day (every file, screenshot, chat, email...) to not only build a knowledge base but also flag what you can proactively be helped with. In the cloud, that's a privacy nightmare + wayy too expensive. On your own silicon, it's private, free, & unlimited.

Pushing what's possible with on-device inference is the core of Sentient OS :D

• Every night at 3 AM, it wakes your Mac and our on-device LLM (Gemma 4 E4B running on a custom fork of LiteRT LM; more below!) reads what's new in your life: files, screenshots (multimodal!), WhatsApp, iMessage, and Apple Notes decoded straight out of the local databases, plus email. It creates a triages out junk / sensitive stuff, and summarizes each item.

And Sentient finally creates a knowledge base of your entire life! (basically an obsidian vault with folders & MDs), along with stuff we think we can proactively help you with

• So by morning, it has proactively found, researched, and offered to do your busy-work for you through Computer Use! The reply you forgot, drafted (from your personal context!); the subscription renewing tomorrow, caught.
• And Sidekick: click your Mac's notch and say "finish this for me"; computer use does the task in your own apps and browser (and can even click around in your apps in the background while you use your computer!). The Computer Use is also grounded in your knowledge base!

The local stack! :D

• Gemma 4 E4B, multimodal with vision, on a customized LiteRT-LM fork. We use MTP/speculative decoding, flash attention, and smart KV cache reuse!
• We reverse engineered codex cli to make computer use with local models work! :)
• And we had to reverse-engineer macOS power management to reliably wake a lid-closed Mac at 3 AM for inference haha!

Sentient’s custom Gemma 4 E3B does 90% of the compute, and the last 10% needs a “frontier” model. You provide that!

I’d had a lot of fun running Qwen 3.7 35B A3B driving computer use, and for the best performance, Kimi K3 works incredibly well!

You also have the choice of using your own ChatGPT/Codex subscription, or OpenRouter or your own endpoint if you wanna use that for the 10% frontier compute. We even have built in first-party support for LM studio! :)

Privacy, enforced by architecture!

Your raw data never leaves the device; and if you choose to use any cloud endpoint, the "frontier" model only ever sees PII-stripped summaries; no accounts exist anywhere; and the whole stack (app + infrastructure) is fully open-source :D (I love OSS -- some of you may know me as the dev of https://github.com/theJayTea/WritingTools, an OSS port of Apple Intelligence Writing Tools to Windows)!

https://sentient-os.ai

brew install --cask sentient-os-labs/tap/sentient-os

Source (feel free to give us a star! :D): https://github.com/Sentient-OS-Labs/sentient-os

Apple Silicon (M1 or newer), macOS 15+, 8 GB of RAM is enough. Free forever for consumer! :D

Would love to share more about my local LLM computer use evals! I’ve found that Kimi K3 works crazy well, while Qwen 3.7 35B A3B can only do super simple tasks.

And lmk if y’all have any cool model recs to try with computer use! :D