A year ago you told me my open-source screen-watching app was flaky. You were right, so I spent the year fixing it with your feedback. Thank you r/LocalLLaMA c: !!

TL;DR: This post is part update, mostly thank you for your support :)) Observer is an open-source app that lets local LLMs watch your screen and notify you (WhatsApp/SMS/email/Discord) when something happens. A year of your feedback later, setup went from "flaky and very manual" (your words, and fair hahaha) to literally typing one sentence and the app auto-controlling itself!!

Hey r/LocalLLaMA!

A year ago I launched Observer here and a LOT of you tried it. The consensus was "cool idea with potential, but flaky and manual" and you were right!

So before anything else: THANK YOU!! :)) and here's what a year of your feedback and support turned into:

- The project is now a SignPath Foundation FOSS project!! This was because of all of you supporting the FOSS development on github!! https://signpath.org/projects !!

- It's now suuuper easy to use making it accessible to everyone!! Here's an example:

Setting it up a year ago:

1. Create an agent in the agent builder (manual creation was even more steps!)
2. Whitelist your phone number (anti-spam, only for SMS/WhatsApp/calls)
3. Select your screen
4. Crop the region you want monitored
5. Start the agent

Now:

1. Type: "Send me a whatsapp when my Steam game finishes downloading, here's my number"
2. Whitelist your number

...and you're done! The agent builder drives the app itself. Watching it configure itself feels like a self-driving car hahahaha.

And the local model can run directly in your browser (transformers.js, no install at all), or point it at whatever you already run: Ollama, bundled llama.cpp in the desktop app, vLLM, any OpenAI-compatible endpoint.

And i'm suuuper grateful that you used it, so here's some cool stuff YOU built:

- u/L0TUSR00T built a person tracker within the framework, crazy sheet!!
- u/sixx7 got banned from a site for "bot activity" (hah), so he put Observer in a VM permanently watching the page in a real browser, reporting changes back to his main agent.
- u/TheRealMasonMac mapped out ADHD/mental-health uses: distraction alerts, hyperfocus check-ins, spiral detection, this comment genuinely shaped the roadmap (added the overlay because of it)
- u/xXG0DLessXx monitors a dumb solar battery's screen and gets alerts when it loses power
- u/philosophissima wanted a ping every time their violin drifts out of tune (looking at the tuner)

But honestly, 95% of use cases are still just "watch this screen and tell me when something happens." Everyone has their one thing they want monitored, and that's the product: not any specific use case, being able to monitor *your* thing.

Some FAQ for transparency (because I know you guys will ask, and should):
- Inference: 100% local if you want. Browser, bundled llama.cpp, or your own server.
- Your screen data never leaves your machine. That's the entire point of the project.
- Browser/system notifications: fully local. WhatsApp/SMS/phone calls go through an API I host, because Twilio isn't free, that part is optional convenience, and whitelisting is my anti-spam guard for it. Telegram/email I provide for free at a loss, Discord goes directly from the app to discord's servers.
- The conversational agent builder uses a cloud model as default; can be changed to a powerful local one.
- License: AGPLv3 (was MIT early on), so forks and improvements stay open-source. Still fully self-hostable with a completely offline option.

Links!
Github (FOSS): https://github.com/Roy3838/Observer
WebApp (Try without any install!): https://app.observer-ai.com/
Discord: https://discord.gg/wnBb7ZQDUC

I'll hang out here in the comments if anyone has a suggestion or a question!