Strix Halo or GPUs?
Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.
I want to build my own AI server, I already have multiple servers at home but none have GPUs neither are powerful enough to host +4B models.
I'd like to be able to host dense 27-30b parameters models, or some MoE with 3b activated parameters.
Let's say I could spend about 2k, what would be the best route? And what tokens speeds should I expect?
[link] [comments]
More from r/LocalLLaMA
-
I tracked EU GPU prices across 15 stores for 50+ days - RTX 5090 is the only card not dropping in price
May 14
-
MLX 16/8/4/2-bit quants of nvidia/llama-embed-nemotron-8b
May 14
-
The RTX 5000 PRO (48GB) arrived and it is better than I expected.
May 14
-
When is Andrej Karpathy going to look at a chicken nugget and tweet that it helped him solve AGI, which in turn inspires 6 random devs to create GitHub projects giving us actual AGI?
May 14
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.