Prismix
OpenAI Fine-Tuning 9 min read

OpenAI Fine-Tuning Guide 2025: Custom Models with Your Data

A practical guide to fine-tuning GPT models on your own data — from deciding when to fine-tune to training jobs, evaluation, and production deployment.

Fine-Tune vs RAG vs Few-Shot: Decision Matrix

Approach When to use Cost
Few-shot prompting First choice — try 5-20 examples in the prompt Zero setup cost
RAG Live or private knowledge needed Vector DB + embed cost
Fine-tuning Consistent tone/format at scale, prompt compression Training fee + lower inference

Fine-tuning pays off when your prompts include many examples (running up token costs) or when prompt-based quality is inconsistent despite optimization. See also: LangChain guide for RAG setup, Pinecone guide for vector storage.

Training Data Format (JSONL)

Each line in your JSONL file must be a valid JSON object with a messages array following the Chat Completions format:

training_data.jsonl — one JSON object per line

{"messages": [{"role": "system", "content": "You extract product names and prices from text."}, {"role": "user", "content": "Apple iPhone 16 Pro Max 256GB for $1199"}, {"role": "assistant", "content": "{\"product\": \"iPhone 16 Pro Max 256GB\", \"price\": 1199}"}]}
{"messages": [{"role": "system", "content": "You extract product names and prices from text."}, {"role": "user", "content": "Samsung Galaxy S25 Ultra (512GB) costs $1299.99"}, {"role": "assistant", "content": "{\"product\": \"Samsung Galaxy S25 Ultra 512GB\", \"price\": 1299.99}"}]}

Quality over quantity: OpenAI requires 10+ examples but recommends 50-100+. Use openai tools fine_tunes.prepare_data -f training_data.jsonl to validate formatting before uploading.

Python Quickstart: Upload Data and Start Training

pip install openai

from openai import OpenAI
import time

client = OpenAI()  # Uses OPENAI_API_KEY env var

# Step 1: Upload training file
with open("training_data.jsonl", "rb") as f:
    upload = client.files.create(file=f, purpose="fine-tune")

print(f"File uploaded: {upload.id}")

# Step 2: Create fine-tuning job
job = client.fine_tuning.jobs.create(
    training_file=upload.id,
    model="gpt-4o-mini",          # Recommended: cheapest + fast
    hyperparameters={
        "n_epochs": 3,             # Default is "auto" — set explicitly for control
        "batch_size": "auto",
        "learning_rate_multiplier": "auto"
    }
)

print(f"Job created: {job.id}")

# Step 3: Poll until done
while True:
    status = client.fine_tuning.jobs.retrieve(job.id)
    print(f"Status: {status.status}")
    if status.status in ("succeeded", "failed"):
        break
    time.sleep(30)

print(f"Fine-tuned model: {status.fine_tuned_model}")

Monitoring Job Progress

Jobs typically run 10-60 minutes depending on dataset size. Watch the training loss in events: 

# Stream events as they arrive
for event in client.fine_tuning.jobs.list_events(job_id=job.id, limit=20):
    print(event.created_at, event.message)

# Useful fields in the status object
status = client.fine_tuning.jobs.retrieve(job.id)
print(status.trained_tokens)           # Total tokens used in training
print(status.error)                    # Non-null if job failed
print(status.result_files)             # Result JSONL with per-step loss

Download the result file and plot training/validation loss. If both curves are decreasing together, training is healthy. If validation loss rises while training loss falls, you are overfitting — reduce n_epochs.

Using Your Fine-Tuned Model

The fine-tuned model works identically to the base model — just swap in the new model ID: 

fine_tuned_model = "ft:gpt-4o-mini:your-org:my-product-extractor:abc123"

response = client.chat.completions.create(
    model=fine_tuned_model,
    messages=[
        {"role": "system", "content": "You extract product names and prices."},
        {"role": "user", "content": "Sony WH-1000XM5 headphones at $349"}
    ]
)
print(response.choices[0].message.content)
# Output: {"product": "Sony WH-1000XM5", "price": 349}

Pricing

Model Training / 1M tokens Inference input Inference output
GPT-4o Mini FT $8.00 $0.30 $1.20
GPT-3.5 Turbo FT $8.00 $3.00 $6.00
GPT-4o FT $25.00 $3.75 $15.00

A 100-example dataset averaging 200 tokens per example = 20,000 training tokens × 3 epochs = 60,000 tokens. At $8/1M that is $0.48 to train. GPT-4o Mini FT is the right starting point for most production fine-tunes.

Monitor OpenAI API Status

Fine-tuning jobs and inference for custom models depend on OpenAI's platform. Prismix tracks live OpenAI status so you know when the API is degraded.

Check OpenAI Status →