Fast little local memory retriever for Hermes
Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.
As title says. Looking for suggestions of a good memory retriever (for use with hindsight/hermes) ideally that can run on a strix halo NPU. GPT OSS 20B would be good based on their outdated rankings but it’s slow on the NPU for this type of task — needs very high throughput to be pulling memories. Anyone else looking to
optimize their agent subtasks with small
models (Bonsai 1 bit? LFM?) let me know your thoughts!
[link] [comments]
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.