Prismix
Replicate ML Hosting 8 min read

Replicate Guide 2025: Run Open-Source AI Models via API

A practical guide to Replicate for developers who want to run Stable Diffusion, FLUX, Llama, Whisper, and thousands of other open-source models without managing infrastructure.

What Is Replicate?

Replicate is a managed platform that hosts open-source machine learning models and exposes them through a unified REST API. Instead of provisioning GPUs, installing CUDA drivers, and managing model weights yourself, you call Replicate's API with your inputs and get back outputs within seconds.

The model catalog includes tens of thousands of community-contributed models across image generation (FLUX, Stable Diffusion XL, SDXL-Turbo), language (Llama 3, Mistral, CodeLlama), speech (Whisper, Tortoise TTS), video (AnimateDiff), and more. All models have a web UI for testing and an auto-generated API.

Pricing is per GPU-second — you pay only when a model runs. No monthly fees, no infrastructure management.

Python SDK Quickstart

# Install

pip install replicate

# Set API token (get from replicate.com/account)

import os
os.environ["REPLICATE_API_TOKEN"] = "r8_YOUR_TOKEN"

import replicate

# Run FLUX image generation
output = replicate.run(
    "black-forest-labs/flux-schnell",
    input={
        "prompt": "A photorealistic cat sitting on a server rack",
        "num_outputs": 1,
        "aspect_ratio": "1:1",
    }
)
print(output)  # List of image URLs

# Run Llama 3 text generation with streaming
for event in replicate.stream(
    "meta/meta-llama-3-8b-instruct",
    input={"prompt": "Explain vector databases in one paragraph."},
):
    print(str(event), end="", flush=True)

# Run Whisper transcription
audio_output = replicate.run(
    "openai/whisper",
    input={"audio": open("audio.mp3", "rb")}
)
print(audio_output["transcription"])

Get your API token: Sign up at replicate.com, go to Account Settings, and copy your API token. New accounts receive free trial credits.

Running via curl

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"input": {"prompt": "a cat on Mars"}}' \
  https://api.replicate.com/v1/models/black-forest-labs/flux-schnell/predictions

The response includes a prediction id and a urls.get field. Poll that URL until status is succeeded. The Python SDK handles this polling automatically.

Deployments: Auto-Scaling & Always-On

Deployments are your own private, auto-scaling instances of any Replicate model. Key benefits over the default shared API:

Eliminate cold starts

Set min_instances=1 to keep a warm instance always running. Cold starts disappear — requests respond in milliseconds.

Auto-scaling

Set max_instances to control how many concurrent predictions can run. Replicate automatically scales between min and max based on load.

Webhooks

Configure a webhook URL on your deployment to receive prediction status events (started/output/logs/completed) via POST request — no polling needed.

Building Custom Models with Cog

Cog is Replicate's open-source tool for packaging ML models. It wraps your model in a Docker container with a standard API.

# predict.py — define your model's interface

import cog
from cog import BasePredictor, Input, Path

class Predictor(BasePredictor):
    def setup(self):
        # Load model weights once at startup
        import torch
        self.model = torch.load("model.pt")

    def predict(self, prompt: str = Input(description="Text prompt")) -> str:
        return self.model.generate(prompt)

# Push to Replicate

cog push r8.im/your-username/your-model

Replicate vs Alternatives

Platform Focus Pricing model Best for
Replicate Open-source model catalog Per GPU-second Any open-source model, rapid prototyping
Hugging Face Model hub + Spaces Free tier + $9/mo Pro Hosting + sharing, community models
Together AI Fast LLM inference Per token High-throughput language models
Modal Serverless GPU compute Per GPU-second Custom code, fine-tuning, batch jobs

See also: Hugging Face guide, Together AI guide, Stable Diffusion guide, and FLUX guide.

Monitor Replicate Status

Replicate outages cause stuck predictions and failed pipelines. Prismix tracks live Replicate API status and sends instant alerts.

Check Replicate Status →