Gemini API 500 Internal Error — How to Diagnose and Fix It

Before writing retry logic, find out whether Gemini is having a platform-wide problem or whether your 500s are isolated. The fastest way is the Prismix status dashboard, which polls the Gemini API and aggregates incident reports in real time.

Both 500 and 503 are server-side and retriable. Wrap your call in a loop that catches 5xx errors, waits an exponentially growing interval with jitter, and gives up after a few attempts. In the current google-genai SDK, 5xx failures raise errors.ServerError (a 429 raises errors.ClientError instead, so this loop deliberately does not retry it):

# Python — google-genai SDK with backoff on 5xx (500 / 503)
import os, time, random
from google import genai
from google.genai import errors

client = genai.Client(api_key=os.environ["GEMINI_API_KEY"])

def generate_with_backoff(max_attempts: int = 6, **kwargs):
    """Retry Gemini 500 / 503 server errors with exponential backoff + jitter."""
    for attempt in range(max_attempts):
        try:
            return client.models.generate_content(**kwargs)
        except errors.ServerError as e:      # 5xx: 500 internal, 503 overloaded
            if attempt == max_attempts - 1:
                raise
            wait = min(60.0, (2 ** attempt) + random.uniform(0, 1))
            print(f"Gemini {e.code} {e.message} — retry {attempt + 1}/{max_attempts - 1} in {wait:.1f}s")
            time.sleep(wait)

resp = generate_with_backoff(
    model="gemini-2.5-flash",
    contents="Explain HTTP 500 in one sentence.",
)
print(resp.text)

// TypeScript — @google/genai with backoff on 5xx
import { GoogleGenAI } from '@google/genai';

const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });

async function generateWithBackoff(params: any, maxAttempts = 6) {
  for (let attempt = 0; attempt < maxAttempts; attempt++) {
    try {
      return await ai.models.generateContent(params);
    } catch (err: any) {
      const status = err?.status ?? 0;        // 500 / 503 are retriable
      if (status < 500 || status >= 600 || attempt === maxAttempts - 1) throw err;
      const wait = Math.min(60_000, (2 ** attempt) * 1000 + Math.random() * 1000);
      console.warn(`Gemini ${status} — retry ${attempt + 1} in ${(wait / 1000).toFixed(1)}s`);
      await new Promise(r => setTimeout(r, wait));
    }
  }
}

const resp = await generateWithBackoff({
  model: 'gemini-2.5-flash',
  contents: 'Explain HTTP 500 in one sentence.',
});
console.log(resp.text);

This waits roughly 1 s, 2 s, 4 s, 8 s, 16 s (with jitter) before surfacing the error. That absorbs the vast majority of short 500 and 503 spikes. Because failed 5xx calls are not billed, the extra attempts add no cost.

If backoff does not help and the same request 500s every time while others succeed, the payload is the culprit. A very large prompt, a malformed function_declarations schema, or an odd file / image input can surface as a generic 500. Narrow it down:

• Trim the prompt: cut the input roughly in half and retry to see if size is the trigger.
• Validate tool / function declarations: a bad JSON schema for a declared function often shows up as 500 rather than 400.
• Re-encode file inputs: re-upload or re-encode PDFs, images, or audio; a corrupt or unsupported blob can fail internally.
• Stream long generations so a big response is delivered incrementally instead of one large payload:

# Stream instead of building one large response
for chunk in client.models.generate_content_stream(
    model="gemini-2.5-flash",
    contents=long_prompt,
):
    print(chunk.text, end="")

A 503 "The model is overloaded" is capacity-specific. If gemini-2.5-pro is saturated, gemini-2.5-flash usually has spare capacity and answers immediately. Chain the models so a 503 rolls over to the lighter one:

MODEL_FALLBACK_CHAIN = [
    "gemini-2.5-pro",     # highest quality, first to saturate under load
    "gemini-2.5-flash",   # lighter — usually has spare capacity during 503s
]

def generate_with_fallback(**kwargs):
    for model in MODEL_FALLBACK_CHAIN:
        try:
            return client.models.generate_content(model=model, **kwargs)
        except errors.ServerError as e:
            # 503 = overloaded: roll over to the next model in the chain
            if e.code == 503 and model != MODEL_FALLBACK_CHAIN[-1]:
                print(f"{model} overloaded (503) — falling back to a lighter model")
                continue
            raise

• Different region: on Vertex AI, retrying the request in another region can dodge a localized capacity crunch.
• Off-peak batches: schedule heavy jobs outside peak US daytime hours to reduce the odds of hitting 503.

During a prolonged 500 / 503 incident, retrying every request just queues up a backlog. A circuit breaker stops sending after a failure threshold, waits a fixed recovery window, then probes before resuming. Since failed 5xx requests are not billed, this protects your throughput and latency rather than your bill:

import time
from google.genai import errors

class CircuitBreaker:
    def __init__(self, failure_threshold=3, recovery_timeout=60):
        self.failures = 0
        self.threshold = failure_threshold
        self.timeout = recovery_timeout
        self.opened_at = None
        self.state = "closed"  # closed | open | half-open

    def record_failure(self):
        self.failures += 1
        if self.failures >= self.threshold:
            self.state = "open"
            self.opened_at = time.monotonic()
            print(f"Circuit OPEN — pausing Gemini calls for {self.timeout}s")

    def record_success(self):
        self.failures = 0
        self.state = "closed"

    def allow_request(self) -> bool:
        if self.state == "closed":
            return True
        if time.monotonic() - self.opened_at >= self.timeout:
            self.state = "half-open"
            return True  # probe request
        return False

breaker = CircuitBreaker()

def safe_generate(**kwargs):
    if not breaker.allow_request():
        raise RuntimeError("Gemini circuit open — skipping request")
    try:
        result = client.models.generate_content(**kwargs)
        breaker.record_success()
        return result
    except errors.ServerError:   # 500 / 503 count toward opening the circuit
        breaker.record_failure()
        raise