Prismix
ChromaDB Vector Database 8 min read

ChromaDB Guide 2025: Open-Source Embedding Database for AI Apps

ChromaDB is the fastest way to add semantic search and RAG to a Python app. It runs in-process — no Docker, no server — and stores embeddings alongside your documents and metadata. Here's everything you need to start building.

What Is ChromaDB?

ChromaDB (GitHub: chroma-core/chroma, 16k+ stars, Apache 2.0) is an open-source AI-native database that stores:

  • Documents — raw text strings
  • Embeddings — float vectors (auto-generated or supplied)
  • Metadata — JSON key/value pairs for filtering
  • IDs — unique string identifiers per document

The same Python API works whether you're running in-process (zero setup), connecting to a local HTTP server, or pointing at ChromaDB Cloud. This makes Chroma the most common starting point for RAG prototypes and the vector store used in LangChain and LlamaIndex tutorials.

Installation

# Core
pip install chromadb

# With OpenAI embeddings
pip install chromadb openai

# With SentenceTransformers (local, no API key)
pip install chromadb sentence-transformers

Requires Python 3.8+. No external service or Docker needed for the default in-process mode. ChromaDB installs a SQLite-backed store and HNSW index entirely in your Python environment.

In-Process vs HTTP Client Modes

EphemeralClient (in-memory)

import chromadb
client = chromadb.EphemeralClient()   # data lost on exit

PersistentClient (disk)

client = chromadb.PersistentClient(path="./chroma_db")  # persists to disk

HttpClient (remote server)

# Start server: chroma run --path /db_path
client = chromadb.HttpClient(host="localhost", port=8000)

Python RAG Quickstart with OpenAI

import chromadb
from chromadb.utils import embedding_functions

# Set up persistent client
client = chromadb.PersistentClient(path="./chroma_db")

# Use OpenAI embeddings
openai_ef = embedding_functions.OpenAIEmbeddingFunction(
    api_key="sk-...",
    model_name="text-embedding-3-small"
)

# Create or load a collection
collection = client.get_or_create_collection(
    name="my_docs",
    embedding_function=openai_ef
)

# Add documents (embeddings auto-generated)
collection.add(
    documents=[
        "Claude is made by Anthropic.",
        "GPT-4o is made by OpenAI.",
        "Gemini is made by Google DeepMind."
    ],
    ids=["doc1", "doc2", "doc3"],
    metadatas=[
        {"company": "anthropic"},
        {"company": "openai"},
        {"company": "google"}
    ]
)

# Semantic search
results = collection.query(
    query_texts=["Which AI is from Anthropic?"],
    n_results=2,
    where={"company": "anthropic"}  # metadata filter (optional)
)

print(results["documents"])

Built-in Embedding Functions

ChromaDB ships with adapters for the most common embedding providers — no manual vectorisation needed:

Function Provider Notes
DefaultEmbeddingFunction all-MiniLM-L6-v2 (local) No API key; 384 dims; good for dev
OpenAIEmbeddingFunction OpenAI text-embedding-3-small (1536 dims) recommended
CohereEmbeddingFunction Cohere embed-english-v3.0 (1024 dims), multilingual option
SentenceTransformerEmbeddingFunction HuggingFace (local) Any SBERT model; best quality/cost for local

ChromaDB vs Pinecone vs Qdrant vs Weaviate

Factor ChromaDB Pinecone Qdrant Weaviate
Setup pip install, zero config Managed cloud only Docker or cloud Docker or cloud
Free self-host Yes (Apache 2.0) No Yes (Apache 2.0) Yes (BSD)
Dev experience Best (in-process) Good (managed) Good (Rust speed) Medium (GraphQL)
Scale ceiling Medium (single node) Very high High High
Best for Local RAG prototypes Production managed RAG Self-hosted production Generative search + GraphQL

See also: Pinecone guide · Qdrant guide · Weaviate guide

Monitor the APIs Your RAG App Depends On

Your ChromaDB RAG app calls OpenAI or Cohere for embeddings. Track uptime for every dependency — get instant alerts when any embedding or inference API degrades.

Monitor AI API Status Free →