Today I learnt the power of LocalLlama

DISCLAIMER: No Ai was prompted in the creation of this post.

Today I had an experience that complely blew my mind, I just had to write it down.

As a bit of background I have been dabbling prompting local models using LM studio for the better part of 18 months now, keeping up to date with the latest releases but never actually doing anything with them.

I am not a software developer, I'm not deeply technical, I wouldnt even call myself a vibe coder.

The last couple weeks I got Hermes connected to my LM studio and it snowballed from there. I migrated to llama.cpp, stood up a dedicated server with some hardware I had lying around to serve some simple services and got everything linked up through Tailscale.

Today I went to a friends house to set her up a little plex server for her and she mentioned her laptop was slow and unusable. Something was seriously wrong as it was taking like 10 minutes to open chrome or file explorer.

I downloaded Pi.dev on this machine, pointed it to my instance of qwen 27b and typed. "This computer is really slow, can you please fix it" and within 5 minutes the computer was fixed, no follow up prompt, and the computer was 5-10x faster.

The issue was simple in hindsight, windows search index was hammering the storage io so hard in the background the computer couldnt do anything else. So Pi turned it off and it was smooth sailing.

Although the action it took was fairly simple, likely trivial to many people here, the UX of it was incredible.

I went to my friends house, pointed my modest little 3090 at her computer and said "Hey please fix this" and it did.

What an incredible time to be alive! I am a true LocalLLama believer now, and I have learnt how much more useful a good harness can make a modest llm.