Prismix
Langfuse LLM Observability Tracing 9 min read

Langfuse Guide 2025: Open-Source LLM Observability & Tracing

Langfuse is the leading open-source platform for LLM observability. It captures traces, spans, and generations for every AI call — tokens, cost, latency, model params — and layers on prompt management, dataset evals, and LLM-as-judge scoring. Self-host it or use the cloud; your data stays under your control.

What Is Langfuse?

Langfuse (MIT license) gives AI engineers full visibility into what their LLM application is doing. Every call is captured as a trace — a tree of spans and generations with timestamps, input/output text, token counts, cost, and model metadata. The dashboard lets you filter, search, and drill into any call within seconds.

Beyond tracing, Langfuse includes: prompt management (versioned prompts fetched at runtime), evaluations (LLM-as-judge + programmatic scoring), and datasets (curate test cases from production traces, run batch evals across model/prompt versions). It competes with LangSmith but is fully open-source and self-hostable.

Install Langfuse SDK

pip install langfuse

Quickstart: First Trace with @observe

Set your API keys (from cloud.langfuse.com or your self-hosted instance), then decorate any function with @observe. Langfuse automatically captures inputs, outputs, and timing.

Set env vars

export LANGFUSE_PUBLIC_KEY="pk-lf-..."
export LANGFUSE_SECRET_KEY="sk-lf-..."
export LANGFUSE_HOST="https://cloud.langfuse.com"  # or your self-hosted URL

first_trace.py

from langfuse.decorators import observe
from langfuse.openai import openai  # drop-in OpenAI wrapper

@observe()
def answer_question(question: str) -> str:
    response = openai.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": question},
        ],
    )
    return response.choices[0].message.content

result = answer_question("What is LLM observability?")
print(result)

That's it. Open the Langfuse dashboard and you'll see the trace — inputs, output, token usage, cost, and latency — with zero additional instrumentation code.

Automatic Tracing: OpenAI, LangChain & LlamaIndex

Langfuse offers drop-in integrations for the most popular frameworks — one import or one callback, and all LLM calls are traced automatically.

OpenAI SDK — drop-in replacement

# Replace this:
# from openai import openai

# With this — all API calls auto-traced:
from langfuse.openai import openai

response = openai.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Explain embeddings."}],
)

LangChain — callback handler

from langfuse.callback import CallbackHandler
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

langfuse_handler = CallbackHandler()

llm = ChatOpenAI(model="gpt-4o-mini")
prompt = ChatPromptTemplate.from_template("Tell me a fact about {topic}")
chain = prompt | llm

response = chain.invoke(
    {"topic": "observability"},
    config={"callbacks": [langfuse_handler]},
)
print(response.content)

LlamaIndex — global handler

from llama_index.core import Settings
from langfuse.llama_index import LlamaIndexCallbackHandler

langfuse_callback = LlamaIndexCallbackHandler()
Settings.callback_manager.add_handler(langfuse_callback)

# All subsequent LlamaIndex queries are now traced automatically
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
docs = SimpleDirectoryReader("data").load_data()
index = VectorStoreIndex.from_documents(docs)
query_engine = index.as_query_engine()
response = query_engine.query("What is RAG?")

Manual Tracing: trace(), span(), generation()

For custom pipelines not covered by automatic integrations, use the low-level SDK to create traces and nest spans manually. This gives full control over trace structure and metadata.

manual_trace.py

from langfuse import Langfuse

langfuse = Langfuse()

# Create a top-level trace (represents one user request)
trace = langfuse.trace(
    name="rag-pipeline",
    user_id="user-123",
    metadata={"environment": "production"},
)

# Retrieval span
retrieval_span = trace.span(
    name="vector-retrieval",
    input={"query": "What is PagedAttention?"},
)
# ... run retrieval ...
retrieval_span.end(output={"chunks": 5, "top_score": 0.92})

# LLM generation
generation = trace.generation(
    name="answer-generation",
    model="gpt-4o-mini",
    model_parameters={"temperature": 0.2, "max_tokens": 512},
    input=[{"role": "user", "content": "What is PagedAttention?"}],
)
# ... call LLM ...
generation.end(
    output="PagedAttention is a memory management algorithm...",
    usage={"input": 45, "output": 89},
)

langfuse.flush()  # Ensure all events are sent before process exits

Prompt Management

Langfuse stores prompts with full version history. Create and edit prompts in the UI, then fetch the latest version at runtime — no code deploys needed to update prompts. Supports A/B testing by linking a trace to the prompt version it used.

Fetch and use a managed prompt

from langfuse import Langfuse
from langfuse.openai import openai

langfuse = Langfuse()

# Fetch the production prompt (latest version by default)
prompt = langfuse.get_prompt("customer-support-system")

# Compile with variables
compiled = prompt.compile(product_name="Pulse", tone="friendly")

response = openai.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": compiled},
        {"role": "user", "content": "How do I reset my password?"},
    ],
    langfuse_prompt=prompt,  # Links this call to the prompt version in the trace
)
print(response.choices[0].message.content)

Fetch a specific version with langfuse.get_prompt("name", version=3) or a label with label="staging" for safe rollout.

Evaluations: LLM-as-Judge & Programmatic Scoring

Langfuse supports two scoring modes: LLM-as-judge evals configured in the UI (no code), and programmatic scores attached to any trace via the SDK.

Programmatic scoring — attach a score to a trace

from langfuse import Langfuse

langfuse = Langfuse()

# After running and evaluating a response:
langfuse.score(
    trace_id="trace-abc123",     # ID from the trace you created
    name="faithfulness",          # Metric name (shows in dashboard)
    value=0.87,                   # 0.0–1.0 or any numeric scale
    comment="Grounded in retrieved context, minor hallucination on date.",
)

# You can also score at the observation (span/generation) level:
langfuse.score(
    trace_id="trace-abc123",
    observation_id="gen-xyz789",
    name="answer-relevance",
    value=4,                       # 1-5 Likert scale
)

For LLM-as-judge: go to Evaluations → Configure in the Langfuse UI, choose a judge model (e.g. GPT-4o), write your evaluation prompt template, and select which traces to score. Results appear alongside your traces automatically.

Datasets: Curate, Evaluate, Compare

Datasets in Langfuse are collections of input/expected-output pairs. You can create them manually, import from CSV/JSON, or add individual traces directly from the dashboard. Then run batch evaluation experiments to compare model versions, prompt versions, or retrieval strategies.

Run a dataset evaluation experiment

from langfuse import Langfuse
from langfuse.openai import openai

langfuse = Langfuse()

# Fetch the dataset
dataset = langfuse.get_dataset("qa-golden-set")

for item in dataset.items:
    # Create an experiment run linked to this dataset item
    with item.observe(run_name="gpt-4o-mini-v2") as trace:
        response = openai.chat.completions.create(
            model="gpt-4o-mini",
            messages=[{"role": "user", "content": item.input["question"]}],
        )
        answer = response.choices[0].message.content

        # Score against expected output
        is_correct = item.expected_output["answer"].lower() in answer.lower()
        langfuse.score(
            trace_id=trace.id,
            name="exact-match",
            value=1.0 if is_correct else 0.0,
        )

langfuse.flush()
print("Experiment complete — view results in Langfuse dashboard")

The dashboard's Experiments view lets you compare runs side-by-side — mean scores, score distributions, and individual trace diffs across model/prompt versions.

Self-Hosting: Docker Compose & Kubernetes

Langfuse is designed to be self-hosted. The stack requires PostgreSQL (trace storage) and ClickHouse (analytics queries). The official docker-compose brings everything up in one command:

Docker Compose quickstart

# Download the official compose file
curl -LO https://raw.githubusercontent.com/langfuse/langfuse/main/docker-compose.yml

# Start all services (Langfuse app + PostgreSQL + ClickHouse)
docker compose up -d

# Open the UI at http://localhost:3000
# Default credentials: create on first sign-in

Kubernetes with Helm

helm repo add langfuse https://langfuse.github.io/langfuse-k8s
helm repo update

# Install with default values (PostgreSQL + ClickHouse as sub-charts)
helm install langfuse langfuse/langfuse \
  --set langfuse.nextauth.secret="your-secret-here" \
  --set langfuse.salt="your-salt-here" \
  --set postgresql.auth.password="pg-password" \
  -n langfuse --create-namespace

Point the SDK at your instance with LANGFUSE_HOST=https://your-domain.com. The self-hosted version has no feature restrictions compared to the cloud offering.

Langfuse vs LangSmith vs Helicone vs Weave

Tool Open-source Self-host Free tier Best for
Langfuse Yes (MIT) Yes 50k obs/mo Open-source, any framework, self-host
LangSmith No No 5k traces/mo Deep LangChain integration
Helicone Yes Yes 100k req/mo Proxy-based, cost analytics, caching
W&B Weave No No Limited ML experiment tracking + LLM evals

See also: LangChain guide · LlamaIndex guide · MLflow guide

Keep Your Entire AI Stack Healthy

Langfuse shows you what your LLM is doing. Prismix shows you when your LLM provider goes down — together they keep your AI app healthy.

Monitor LLM Provider Status →