Prismix
Cloudflare Edge AI 8 min read

Cloudflare Workers AI Guide 2025: Run AI Models at the Edge

A practical guide to Cloudflare Workers AI for developers who want serverless GPU inference without cold starts — right inside their existing Cloudflare stack.

What Is Cloudflare Workers AI?

Cloudflare Workers AI is a managed inference service built into the Cloudflare developer platform. Models run on GPUs distributed across Cloudflare's global network — the same infrastructure that routes 20% of internet traffic. This gives you AI inference close to your users with no cold starts, no GPU provisioning, and no separate inference server to manage.

The integration story is tight: if you already run a Worker, you add an AI binding to wrangler.toml, then call env.AI.run(). No separate API key, no base URL — it's just another binding next to KV, R2, and D1.

Prismix itself runs on Cloudflare Pages + Workers. If you want to see Workers AI in a real production app, you are already looking at one. Browse the MCP directory to find Cloudflare-specific MCP servers.

Available Models

Model ID Type Use case
@cf/meta/llama-3.3-70b-instruct-fp8-fast LLM Chat, summarization, code
@cf/mistral/mistral-7b-instruct-v0.2 LLM Fast chat, lower cost
@cf/google/gemma-2b-it-lora LLM Lightweight tasks
@cf/stabilityai/stable-diffusion-xl-base-1.0 Image Text-to-image generation
@cf/openai/whisper STT Audio transcription
@cf/baai/bge-base-en-v1.5 Embeddings Semantic search, RAG

All model IDs use the @cf/ prefix. Browse the full catalog at developers.cloudflare.com/workers-ai/models.

Workers Setup and AI Binding

Add the AI binding to your wrangler.toml, then use it in TypeScript:

wrangler.toml

name = "my-ai-worker"
main = "src/index.ts"
compatibility_date = "2024-11-04"

[ai]
binding = "AI"

src/index.ts

export interface Env {
  AI: Ai;
}

export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    const response = await env.AI.run(
      "@cf/meta/llama-3.3-70b-instruct-fp8-fast",
      {
        messages: [
          { role: "system", content: "You are a helpful assistant." },
          { role: "user", content: "What is edge computing in one paragraph?" },
        ],
      }
    );

    return Response.json(response);
  },
};

Local dev note: run wrangler dev --remote (not wrangler dev) to use the AI binding. Local mode does not emulate GPU inference.

REST API (No Worker Required)

Call Workers AI directly from any language using your Cloudflare API token and Account ID: 

import requests, os

ACCOUNT_ID = os.environ["CF_ACCOUNT_ID"]
API_TOKEN  = os.environ["CF_API_TOKEN"]

url = f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/meta/llama-3.3-70b-instruct-fp8-fast"

resp = requests.post(
    url,
    headers={"Authorization": f"Bearer {API_TOKEN}"},
    json={
        "messages": [
            {"role": "user", "content": "What is the Cloudflare edge network?"}
        ]
    }
)
print(resp.json()["result"]["response"])

Get your Account ID from the Cloudflare dashboard sidebar. Generate an API token at My Profile > API Tokens with the "Workers AI" permission.

Embeddings and Vectorize Integration

Workers AI embeddings pair directly with Cloudflare Vectorize for a fully edge-native RAG pipeline: 

// Generate embeddings for a document
const embedResult = await env.AI.run(
  "@cf/baai/bge-base-en-v1.5",
  { text: "Cloudflare Workers AI runs at the edge." }
);

// Upsert into Vectorize
await env.VECTORIZE.upsert([{
  id: "doc-1",
  values: embedResult.data[0],
  metadata: { text: "Cloudflare Workers AI runs at the edge." }
}]);

// Query at inference time
const query = await env.AI.run(
  "@cf/baai/bge-base-en-v1.5",
  { text: "What is edge AI?" }
);
const matches = await env.VECTORIZE.query(query.data[0], { topK: 3 });
console.log(matches);

Workers AI vs Replicate vs Together AI

Provider Cold start Free tier Best for
Workers AI None 10k neurons/day CF stack, edge latency
Together AI ~1s warm $5 credit 100+ models, fine-tuning
Replicate 5-30s cold Small credit Custom model deploy (Cog)

Also see: Together AI guide, Replicate guide, OpenAI API guide.

Monitor Cloudflare Workers AI Status

Cloudflare Workers AI shares infrastructure with Cloudflare's global network. Prismix tracks live Cloudflare status and sends alerts when inference is degraded.

Check Cloudflare AI Status →