Claude Context Window Exceeded — Fix "Prompt Is Too Long"

Use the countTokens endpoint (exact, free)

Anthropic provides a token counting endpoint that returns the exact count without consuming API credits or producing output. Add a guard before every expensive call:

import anthropic

client = anthropic.Anthropic()

system = "You are a helpful assistant..."
messages = [
    {"role": "user", "content": "Summarize this 80-page document: ..."},
    {"role": "assistant", "content": "Here is the summary..."},
    {"role": "user", "content": "Now compare it to this other document: ..."},
]

# Count tokens before sending — no credits used
token_count = client.beta.messages.count_tokens(
    model="claude-3-5-sonnet-20241022",
    system=system,
    messages=messages,
)
print(f"Input tokens: {token_count.input_tokens}")  # e.g. 187432

CONTEXT_LIMIT = 200_000
MAX_OUTPUT = 4096

if token_count.input_tokens + MAX_OUTPUT > CONTEXT_LIMIT:
    print("Too long — triggering truncation before send")
    # ...truncate messages (see Fix 2)
else:
    response = client.messages.create(
        model="claude-3-5-sonnet-20241022",
        max_tokens=MAX_OUTPUT,
        system=system,
        messages=messages,
    )

Fast estimation (no API call): when you need a quick check without an extra round-trip, divide character count by 4 for English prose (3 for code-heavy content). This is accurate to within ~10%:

def estimate_tokens(text: str, is_code: bool = False) -> int:
    chars_per_token = 3 if is_code else 4
    return len(text) // chars_per_token

# Estimate total input
total = estimate_tokens(system)
for msg in messages:
    content = msg["content"] if isinstance(msg["content"], str) else ""
    total += estimate_tokens(content) + 4  # 4 overhead per turn

print(f"Estimated tokens: {total}")

Drop oldest turns (sliding window)

The simplest approach: keep only the most recent N turns. Always keep the system prompt intact — only drop message history:

def trim_history(messages: list, max_turns: int = 10) -> list:
    """Keep the last max_turns complete user/assistant pairs."""
    # messages come in pairs: [user, assistant, user, assistant, ...]
    # keep last max_turns pairs = last max_turns * 2 messages
    keep = max_turns * 2
    if len(messages) > keep:
        return messages[-keep:]
    return messages

# Or trim until it fits under token budget
def trim_to_fit(client, model, system, messages, max_tokens_out=4096):
    limit = 200_000 - max_tokens_out
    while len(messages) > 2:
        count = client.beta.messages.count_tokens(
            model=model, system=system, messages=messages
        )
        if count.input_tokens <= limit:
            break
        # Drop oldest pair (user + assistant)
        messages = messages[2:]
    return messages

Compress with a rolling summary

For long-running assistants, periodically ask Claude to summarize old turns, then replace them with a single context message:

def compress_history(client, model, messages, keep_recent=4):
    """Summarize early turns, keep recent turns verbatim."""
    if len(messages) <= keep_recent * 2:
        return messages

    old = messages[:-keep_recent * 2]
    recent = messages[-keep_recent * 2:]

    summary_resp = client.messages.create(
        model=model,
        max_tokens=1024,
        messages=[
            *old,
            {"role": "user", "content": (
                "Summarize the conversation above in 3-5 sentences, "
                "capturing all key facts, decisions, and context."
            )},
        ],
    )
    summary_text = summary_resp.content[0].text

    compressed = [
        {"role": "user", "content": f"[Earlier conversation summary]: {summary_text}"},
        {"role": "assistant", "content": "Understood. I have that context."},
        *recent,
    ]
    return compressed

Prompt caching for repeated context

If your system prompt or reference document is large and stays the same across many requests, enable prompt caching. Cached tokens cost 10% of normal input price on re-use, and the cache lasts 5 minutes. Add cache_control to the last block you want cached:

response = client.beta.prompt_caching.messages.create(
    model="claude-3-5-sonnet-20241022",
    max_tokens=1024,
    system=[
        {
            "type": "text",
            "text": "You are an expert code reviewer...",
        },
        {
            "type": "text",
            "text": entire_codebase_as_string,  # could be 50K+ tokens
            "cache_control": {"type": "ephemeral"},
        },
    ],
    messages=[{"role": "user", "content": "Review the auth module"}],
)

// TypeScript — prompt caching
const response = await client.beta.promptCaching.messages.create({
  model: 'claude-3-5-sonnet-20241022',
  max_tokens: 1024,
  system: [
    { type: 'text', text: 'You are an expert code reviewer...' },
    {
      type: 'text',
      text: entireCodebaseAsString,
      cache_control: { type: 'ephemeral' },
    },
  ],
  messages: [{ role: 'user', content: 'Review the auth module' }],
});

// Check cache usage in response
console.log(response.usage.cache_read_input_tokens);   // tokens served from cache
console.log(response.usage.cache_creation_input_tokens); // tokens written to cache

Extended thinking eats into your context budget

Extended thinking reserves thinking_budget_tokens from the 200K context for internal reasoning. The thinking is not returned as text output, but it does count. If your input is already large, a high thinking budget causes "max_tokens + thinking_budget exceeds context":

# Context budget formula with extended thinking:
# input_tokens + thinking_budget_tokens + max_tokens <= 200_000

input_tokens = 150_000   # large document
thinking_budget = 10_000
max_tokens = 4_096

if input_tokens + thinking_budget + max_tokens > 200_000:
    # Reduce thinking budget
    thinking_budget = 200_000 - input_tokens - max_tokens - 1_000  # 1K safety margin
    thinking_budget = max(1_024, thinking_budget)  # minimum useful budget

response = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=max_tokens,
    thinking={
        "type": "enabled",
        "budget_tokens": thinking_budget,
    },
    messages=[{"role": "user", "content": prompt}],
)

• Reasonable budgets: 1,000–2,000 tokens for straightforward tasks; 4,000–8,000 for complex reasoning; only 10,000+ for the hardest multi-step problems.
• Extended thinking only on claude-opus-4-8 and claude-sonnet-4-6 — it is not available on Haiku models.