Boom Times for Inference Providers?

Less than a year ago, our reporters kept hearing doubts about a group of startups called inference providers. Companies like Fireworks, Baseten and Together AI, which rent out Nvidia servers to app developers and help them customize open-source models, had grown quickly but seemed at risk of getting steamrolled by major cloud providers that could build these capabilities in-house. 

Those traditional cloud providers also have the advantage of owning the AI chips servers they rent out; inference firms, in contrast, generally rent the chips from those traditional providers and then turn around and rent them out to their customers. That dynamic has dragged down the gross profit margins of some inference providers in the past.