I ran the actual break-even math on buying vs renting an H200 box, and it is not where I expected

Every rent-vs-buy thread I read has confident people on both sides, but not many actually show the numbers. So I finally ran the numbers for our own decision. Posting the working here in case it is useful, or feel free to point it out in case someone thinks it's wrong.

An 8-GPU HGX H200 server lands somewhere near $320k-$420k, with roughly $370k being a reasonable midpoint.

On the rental side, the median on demand H200 price across 34 providers was about $4.40/GPU-hour as of September 18. The $2-$3 rates you sometimes see are closer to spot pricing.

Using the $370k as midpoint and a rental equivalent of $35.20/hour, the hardware only break even works out to approximately:

-> 14.4 months at 100% utilisation

-> 24 months at 60% utilisation

-> 36 months at 40% utilisation

Ofc, most small teams with bursty training and steady inference aren't sustaining 100% utlisation.

This is only hardware level comparison. There are at least four other things to include:

Power and cooling (I was quoted more for a colo cage than I had budgeted)

Depreciation (Whatever you assume, halve it. Resale on last-gen datacenter parts is thin)

Your own time.

Idle hours.

My conclusion was that roughly 60% sustained utilisation for 2 years, owning wins. Below 40%, renting wins. You can also sell your idle capacity to offtake networks and offset the cost of your device.

I'd love to hear what utilisation are people here actually seeing?