What to run at 128GB VRAM?
Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.
Long time lurker, but I'm finally upgrading to 128GB VRAM, and I'm trying to figure out what to run. I had been leaning towards Qwen3.8 Flash-Next at ~Q4, and I generally prefer to not run anything below Q4. But I feel like the reception to Flash-Next has been a bit "meh", so I'm considering GLM 5.3 at ~Q2 or Deepseek 4 Flash at Q2 or Q3. I'm sure I'll try all 3, but I'm really curious what people in the same boat have been doing?
Edit: configuration is 2 X CMP 170 HXs (64GB each) + ~256 GB of DDR4 RAM. Spilling into RAM is basically not an option, except for the ngrams and caching
[link] [comments]
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.