Budget llm for chatting and analysing pdf documents
Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.
My dad and I want an llm that can scan pdfs and get the useful data out of it at a reasonable speed and be able to chat with it about those documents.
What kind of hardware would be best for this and what kind of power usage would a machine like this use?
The budget we are currently looking at is between 600-1000€ but it can become more if it needs to be as we don’t want to wait half an hour for a response.
We have currently tried a 16gb m2 Mac mini but that seems to be a bit too slow.
We tested it dockling but that seems to only work for some pages of the pdf and we don’t know if there are any alternatives.
help is very appreciated.
[link] [comments]
More from r/LocalLLaMA
-
I bundled a fully local LLM inside my Unity game. No internet, no cloud, no API key. The conversation is the gameplay.
Jun 8
-
Xiaomi just claimed 1,000+ tps on a 1T model using a standard 8-GPU server
Jun 8
-
Luce Spark: a 35B MoE on a 16 GB GPU, without the offload tax
Jun 8
-
OpenEnv is now owned by HF, Torch, Prime Intellect, Unsloth, Modal, Mercor, and more! Use it for training agents.
Jun 8
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.