GitHub Models Guide 2025: Try & Compare AI Models in GitHub

What Is GitHub Models?

GitHub Models is a model marketplace built into GitHub and powered by Azure AI Foundry. It gives every GitHub user free access to a curated catalog of LLMs — without needing an Azure account, credit card, or separate API key setup. You test models in a browser playground, then call the same models from your code using a GitHub personal access token (PAT) as the API key.

The endpoint is OpenAI-compatible, so you can use the openai Python SDK or any HTTP client that speaks the OpenAI Chat Completions format.

https://github.com/marketplace/models

See also: GitHub status page

Available Models

The catalog changes as Azure AI Foundry adds new models. As of mid-2025, available models include:

Playground UI

Each model page on github.com/marketplace/models includes a playground with:

• System prompt input — set context before the conversation starts.
• Parameter sliders — temperature, max tokens, top-p, frequency penalty.
• Side-by-side comparison — open two models in split view to compare outputs on the same prompt.
• "View code" button — generates ready-to-run Python, JavaScript, or curl code for the current prompt and parameters.

Python SDK Quickstart

The API is OpenAI-compatible. Use your GitHub PAT as the API key and point the base URL at Azure AI's inference endpoint:

pip install openai

import os
from openai import OpenAI

# Use your GitHub Personal Access Token (classic or fine-grained)
# Set GITHUB_TOKEN env var or paste directly
client = OpenAI(
    base_url="https://models.inference.ai.azure.com",
    api_key=os.environ["GITHUB_TOKEN"],
)

response = client.chat.completions.create(
    model="gpt-4o",           # any model from the catalog
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user",   "content": "Explain what a tensor is in 2 sentences."},
    ],
    temperature=0.7,
    max_tokens=256,
)

print(response.choices[0].message.content)

response = client.chat.completions.create(
    model="Meta-Llama-3.1-70B-Instruct",
    messages=[...],
)

Rate Limits & Going to Production

The free tier is for development and testing. Rate limits by tier:

For production, switch to an Azure AI Foundry endpoint. The code is identical — swap base_url and replace the GitHub token with an Azure API key. No model name changes needed.

# Dev (GitHub Models)
client = OpenAI(
    base_url="https://models.inference.ai.azure.com",
    api_key=os.environ["GITHUB_TOKEN"],
)

# Production (Azure AI Foundry)
client = OpenAI(
    base_url="https://YOUR-ENDPOINT.openai.azure.com/",
    api_key=os.environ["AZURE_API_KEY"],
)

GitHub Models vs GitHub Copilot vs Azure OpenAI

See also: GitHub Copilot guide · OpenAI API guide · GPT-4o guide