Got Really lucky and need your advice
Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.
So, I got the chance to get either a rig of like 8 RTX PRO 6000s or the GB300. Which should I take? Its gonna be used by like 10 people, but im the primary user.
Edit: Thought I'd add some context: The RTX6000s would be PCIe Boards. So if I shard the model across the GPU then the effective bandwidth drops to 64gb/s. The GB300 is a unified HBM memory at 252 GB, thats like 7TB/s.
[link] [comments]
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.