built a memory pipeline on Qwen3 235B A22B Instruct 2507 that scored #1 on LongMemEval-S (470/500) while being ~10x more token efficient than the next best system

Over past ~10 months I've been iterating on my memory system so I can make a proper assistant, like Rick's garage from Rick and Morty. I benched my latest iteration and it scored top out of any system I know of (470/500 on LongMemEval-S), while being way more token efficient and cheaper.

The entire pipeline, storage and retrieval, runs on Qwen3 235B A22B Instruct 2507 with user chosen model as the answering model. The hardest bit wasn't even coming up with the architecture, it was making the system reliable, as I found so many inexplicable errors coming up from these smaller models. I would say like 90% of my time went into prompt tweaking and guards to make this work😭😭 It's a great model, don't get me wrong, but sometimes so annoying to work with.

I did a writeup here: https://c137.ai/research/overhaul and I have a bench viewer here to see all questions and prompts per model I ran: https://c137.ai/research/bench-viewer and finally, I open sourced a repo where you can rerun against my prompts and grade using official grader to confirm scores aren't fudged: https://github.com/ra1ngod/c137-runner

I run different prompts in prod because these prompt scaffolds were very gemini tuned😭😭, the bench ran on those exact prompts shown in viewer and runner repo. I didn't put the gold in any prompts though; I always used generalised or adjacent examples and you can confirm this as all prompts are there.

The write up is like ~20 mins to read so if you have any questions I can answer here to save you time. If anyone decides to try out the app as well (free to try obviously), any feedback is much appreciated. If you DM me I am happy to explain any features, how to import existing chats, resolve any issues you run into or add any features you want.