Prismix
Together AI Open-Source Models 7 min read

Together AI Guide 2025: Fast Open-Source Model Inference

Together AI gives you access to 100+ open-source models via an OpenAI-compatible API — often at 5-10x lower cost than proprietary alternatives. Drop-in replacement for OpenAI with Llama, DeepSeek, Mixtral, FLUX, and more.

What Is Together AI?

Together AI (together.ai) is a cloud inference platform focused on open-source AI models. It provides:

  • 100+ hosted models — LLMs, vision models, image generators, embedding models
  • OpenAI-compatible REST API (same SDK, different base_url)
  • Serverless inference with per-token pricing (no reserved capacity needed)
  • Dedicated endpoints for high-volume production workloads
  • Fine-tuning API for Llama, Mistral, and other open-weight models
  • Embeddings endpoint for RAG pipelines

Together AI raised $102M in 2024 and is used by AI startups and researchers for cost-effective open-source inference. Check live status at prismix.dev/service/togetherai.

Python Quickstart (OpenAI SDK Drop-In)

pip install openai  # same SDK, no extra package needed
from openai import OpenAI

# Only two changes from OpenAI: base_url + api_key
client = OpenAI(
    base_url="https://api.together.xyz/v1",
    api_key="your_together_api_key",  # from api.together.xyz/settings/api-keys
)

response = client.chat.completions.create(
    model="meta-llama/Llama-4-Scout-17B-16E-Instruct-Turbo",
    messages=[
        {"role": "user", "content": "Explain vector databases in 3 sentences."}
    ],
    max_tokens=200,
)
print(response.choices[0].message.content)

# Streaming works identically
stream = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-V3",
    messages=[{"role": "user", "content": "Write a haiku about Python."}],
    stream=True,
)
for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

Available Models

Model Model ID Price (per 1M tokens)
Llama 4 Scout 17B meta-llama/Llama-4-Scout-17B-16E-Instruct-Turbo ~$0.18
DeepSeek-V3 deepseek-ai/DeepSeek-V3 ~$1.25
Llama 3.3 70B Turbo meta-llama/Llama-3.3-70B-Instruct-Turbo ~$0.88
Mixtral 8x22B mistralai/Mixtral-8x22B-Instruct-v0.1 ~$1.20
Qwen 2.5 72B Qwen/Qwen2.5-72B-Instruct-Turbo ~$1.20
FLUX.1 Schnell (image) black-forest-labs/FLUX.1-schnell-Free Free tier available

Model IDs are case-sensitive. List all models via GET https://api.together.xyz/v1/models.

Image Generation (FLUX.1)

import requests

response = requests.post(
    "https://api.together.xyz/v1/images/generations",
    headers={"Authorization": "Bearer YOUR_KEY"},
    json={
        "model": "black-forest-labs/FLUX.1-schnell-Free",
        "prompt": "A futuristic data center with glowing servers",
        "width": 1024,
        "height": 768,
        "steps": 4,
        "n": 1,
    }
)
image_url = response.json()["data"][0]["url"]

Together AI vs Groq vs Fireworks AI vs Replicate

Factor Together AI Groq Fireworks AI Replicate
Speed (tokens/sec) 100-300 500-800+ 100-400 50-200
Model count 100+ ~25 50+ Thousands
Fine-tuning Yes No Yes Via Trainings API
Image generation Yes (FLUX.1) No Yes (FLUX.1) Yes (many models)
Embeddings Yes No Yes Via individual models
Best for Breadth + fine-tuning Max speed (LPU) Speculative decoding Niche / diffusion models

See the full speed comparison at Groq vs Together AI or explore Groq Guide and Llama Guide for model-specific guidance.

Monitor Together AI & Your Inference Stack

Track Together AI uptime alongside Groq, Fireworks, and OpenAI from one dashboard. Get instant alerts when your inference provider degrades.

Check Together AI Status →