Hey Reddit. This is a post asking for advice / user experience.

The goal is simple: deploy a small private server for a developer to run a harness that rivals/beats Claude Opus (in perf/intelligence, not necessarily speed).

I believe the model to target is a Q3 or so version of the Qwen 3.8 Next model, and to build a dual GPU, 64GB setup. Streaming n-gram off RAM/SSD (~85GB for total model + KV, minus 27 n-gram = 63GB) would mean the 64GB might be enough.

Budget: as low as possible, up to $4000. I am willing to get extremely scrappy though.

I am wondering: if I want to run this model with a full 256k context, and want about ~500 tok/sec PP, and +20 tok/sec decode, what kind of cards do I need?

I see my options as follows:

Budget and weird

Something super silly with 2 or 3 Tesla V100s, the 32GB version, which I assume is possible but have yet to see anyone actually deploy. This ofc would have the pro of being cheap, the con of being ancient and losing community support as even better models come out. It would also mean relying on a more serious mobo with proper PCIe lanes.
Mobo and parts would be maybe $600, 2 cards on eBay ~$2000. Doable.

Budget and interesting

AMD Strix 64GB edition (way cheaper, around $2000) and a 5060 Ti 16GB, then run the model on Vulkan. This would mean we could utilize the iGPU and the fast RAM to handle the experts and have the model attention running on the 5060 Ti — this sounds like the best of both worlds.

Costs $2600 + $750 + OcuLink riser = $3500. Doable, and would probably be very fast.
Cons: running on Vulkan means slower PP, weird deployment.

Budget and Intel

2x Arc Pro B70s.
I have seen vLLM has a fork that plays nice on Intel. Going with vLLM over llama.cpp means that offloading the n-gram would be more finicky — so I would probably need 64GB of RAM. That's $400 today, so system price would go up significantly. And they aren't that fast — from what I can see anyways, the VRAM is about 2x the Strix Halo in Gbps.
$1200 x2 + $800 for system = $3200. Con: probably slower.

Not So Budget and Modern ROCm

2x the RX 7900 XTX or r9700. Better support than Intel, probably could offload to SSD properly and use this more modern + newer models down the line. Fastest, most future proof.
Costs $1700 x2 + $500 ish = $4100 overall.

I'm looking for any benchmarks of these systems, advice and/or anecdotes from anyone that wants to provide their two cents.

Note that this model seems like a good target, but ofc this system would be meant to continually provide a server for a single dev with modern, frontier-matching AI without breaking the bank on a 5090 or something.

Going forward, I expect the ~mid-tier, Opus-class of models to be what people actually can afford to run, and with optimizations like n-gram I assume they will be runnable on even less RAM as time goes on.

Obviously, prices are getting crazy, but they seem to just keep rising — so let me know all/any thoughts.

also - this isnt written by AI - so pls read.