r/LocalLLaMA · · 1 min read

Got Really lucky and need your advice

Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.

So, I got the chance to get either a rig of like 8 RTX PRO 6000s or the GB300. Which should I take? Its gonna be used by like 10 people, but im the primary user.

Edit: Thought I'd add some context: The RTX6000s would be PCIe Boards. So if I shard the model across the GPU then the effective bandwidth drops to 64gb/s. The GB300 is a unified HBM memory at 252 GB, thats like 7TB/s.

submitted by /u/Amos-Tversky
[link] [comments]

Discussion (0)

Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

Sign in →

No comments yet. Sign in and be the first to say something.

More from r/LocalLLaMA