LANGCHAIN_API_KEY vs LANGSMITH_API_KEY — which to use?

Both LANGCHAIN_API_KEY and LANGSMITH_API_KEY work — LANGSMITH_API_KEY is the newer canonical name. The SDK checks both. Key format: starts with ls__ (two underscores). Generate at smith.langchain.com settings. LANGCHAIN_ENDPOINT (default https://api.smith.langchain.com) and LANGCHAIN_PROJECT are the other two key env vars.

LangSmith evaluation not working — how to fix?

Evaluation checklist: (1) evaluator function must return dict: {'score': 0.9, 'comment': 'good'}; (2) dataset must have at least one example; (3) wrap evaluator in try/except to prevent one failure from cancelling all; (4) test your chain independently before evaluation; (5) lower max_concurrency in evaluate() if hitting rate limits; (6) use target=your_chain_function, not the compiled chain object.

LLM Observability 5 min read

LangSmith Not Working?

Traces not appearing (LANGCHAIN_TRACING_V2=true missing), API 401, wrong project, evaluation stuck, or dataset upload failing? Check live status and fix step by step.

LangSmith — live status

Updated every 5 minutes. Full history at prismix.dev/service/langsmith.

Full status →

What's wrong? Diagnose fast

📊

Traces not appearing (most common)

LANGCHAIN_TRACING_V2=true is NOT set — this is the #1 cause. Must be set in the same process/environment as your LangChain code. Verify: import os; print(os.environ.get("LANGCHAIN_TRACING_V2")). Also need LANGCHAIN_API_KEY=ls__YOUR_KEY. Traces buffer 5-30s before appearing.

🔑

API 401 — key format and variable name

LangSmith key starts with ls__ (two underscores, not one). Set as LANGCHAIN_API_KEY or LANGSMITH_API_KEY (both work — LANGSMITH_API_KEY is newer canonical name). Generate at smith.langchain.com/o/ORG/settings/api-keys. SDK: Client(api_key="ls__YOUR_KEY").

📁

Wrong project / traces in wrong place

Set LANGCHAIN_PROJECT=my-project-name. If not set, traces go to "default" project. Project is created automatically on first trace. Project names are CASE-SENSITIVE. Per-run project: with trace(name="run", project_name="my-project"): or @traceable(project_name="my-project").

🧪

Evaluation stuck or returning no results

Evaluator function MUST return dict: {"score": 0.9} or {"key": "correctness", "score": 1, "comment": "good"}. Common error: returning None or a plain float. If one example fails, evaluation continues — check individual example logs. Lower max_concurrency if hitting rate limits.

📄

Dataset upload failing

Dataset format: list of dicts with "inputs" and "outputs" keys. Example: [{"inputs": {"question": "..."}, "outputs": {"answer": "..."}}]. CSV upload also works (column headers become input/output keys). Max 10k examples per upload. Rate limit: 100 examples/sec — batch large uploads.

📦

langchain-core version conflict

LangSmith tracing is bundled in langchain-core ≥0.1.0. If traces are not appearing after setting env vars, check: pip show langchain-core. If below 0.1.0, upgrade: pip install langchain --upgrade langchain-core --upgrade langsmith --upgrade. Conflicts happen when mixing old langchain + new langsmith packages.

LangSmith setup quick reference

Required environment variables

# .env or your shell — ALL FOUR must be set for tracing to work
LANGCHAIN_TRACING_V2=true               # <-- most commonly missing
LANGCHAIN_API_KEY=ls__YOUR_KEY          # or LANGSMITH_API_KEY=ls__YOUR_KEY
LANGCHAIN_PROJECT=my-project-name       # optional, defaults to "default"
LANGCHAIN_ENDPOINT=https://api.smith.langchain.com  # optional, for EU or self-hosted

Python — enable tracing + verify

import os
from dotenv import load_dotenv
load_dotenv()

# Verify env vars are set
print(os.environ.get("LANGCHAIN_TRACING_V2"))  # should print: true
print(os.environ.get("LANGCHAIN_API_KEY", "NOT SET"))  # should print: ls__...

# Now run your LangChain code — traces appear in smith.langchain.com
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(model="gpt-4o")
result = llm.invoke("Hello!")  # this trace appears in LangSmith

Custom evaluator that works

from langsmith.evaluation import evaluate

def correct_answer_evaluator(run, example):
    """MUST return dict with 'score' key."""
    predicted = run.outputs.get("answer", "")
    expected = example.outputs.get("answer", "")
    correct = predicted.strip().lower() == expected.strip().lower()
    return {"key": "correctness", "score": int(correct)}  # NOT return correct

results = evaluate(
    your_chain.invoke,        # callable, not the chain object
    data="my-dataset-name",   # dataset name in LangSmith
    evaluators=[correct_answer_evaluator],
    max_concurrency=4,        # lower if hitting rate limits
    experiment_prefix="run-1",
)

Step-by-step fix

1

Check live LangSmith status

Visit prismix.dev/service/langsmith. Also smith.langchain.com/status for the official status page.
2

Set LANGCHAIN_TRACING_V2=true (most common fix)

This is the single most common cause of missing traces. Must be set as an environment variable, not just in code. In Python: os.environ["LANGCHAIN_TRACING_V2"] = "true" before any LangChain imports, OR set in .env and call load_dotenv() first.
3

Fix API key (ls__ prefix)

Key format: ls__xxxxx (two underscores). Set as LANGCHAIN_API_KEY or LANGSMITH_API_KEY. Generate at smith.langchain.com → Settings → API Keys.
4

Fix project routing

Set LANGCHAIN_PROJECT=my-project. Projects are created automatically. For per-run routing: @traceable(project_name="my-project") decorator or use context manager.
5

Fix evaluation

Evaluator must return {"key": "name", "score": 0.9}. Test one example before running full evaluation. Wrap in try/except to prevent one failure stopping all evaluations. Pass max_concurrency=1 for debugging.

🔔

Get alerted when LangSmith goes down

Star LangSmith on Prismix and get emailed the moment status changes. Free, no credit card.