Site Logo

Mastering Similarity Search with LangChain’s Chroma Vector Store

Introduction

In the rapidly evolving landscape of artificial intelligence, efficiently retrieving relevant information from large datasets is a cornerstone for applications such as semantic search, question-answering systems, recommendation engines, and conversational AI. LangChain, a versatile framework for building AI-driven solutions, integrates the Chroma vector database to provide a high-performance vector store for similarity search. This comprehensive guide delves into the Chroma vector store’s setup, core features, performance optimization, practical applications, and advanced configurations, equipping developers with detailed insights to build scalable, context-aware systems.

To understand LangChain’s broader ecosystem, start with LangChain Fundamentals.

What is the Chroma Vector Store?

LangChain’s Chroma vector store leverages Chroma, an open-source, lightweight, and embeddable vector database designed for fast similarity search on high-dimensional vector embeddings. Chroma enables developers to index, store, and query embeddings—numerical representations of text or data—efficiently, making it ideal for tasks requiring semantic understanding, such as retrieving documents conceptually similar to a query. The Chroma vector store in LangChain, provided via the langchain-chroma package, offers a seamless interface with support for persistent storage, metadata filtering, and hybrid search capabilities.

For a primer on vector stores, see Vector Stores Introduction.

Why Chroma?

Chroma stands out for its simplicity, speed, and flexibility, supporting both in-memory and persistent storage with minimal setup. It handles millions of vectors with low latency, integrates easily with Python ecosystems, and supports advanced features like metadata filtering and approximate nearest-neighbor search. LangChain’s implementation abstracts Chroma’s complexities, making it a go-to choice for AI applications.

Explore Chroma’s capabilities at the Chroma Documentation.

Setting Up the Chroma Vector Store

To use the Chroma vector store, you need an embedding function to convert text into vectors. LangChain supports providers like OpenAI, HuggingFace, and custom models. Below is a basic setup using OpenAI embeddings with an in-memory Chroma instance:

from langchain_chroma import Chroma
from langchain_openai import OpenAIEmbeddings

embedding_function = OpenAIEmbeddings(model="text-embedding-3-large")
vector_store = Chroma(
    collection_name="langchain_example",
    embedding_function=embedding_function
)

This initializes a Chroma vector store in memory with a collection named langchain_example. The embedding_function generates vectors (e.g., 1536 dimensions for OpenAI’s text-embedding-3-large).

For alternative embedding options, visit Custom Embeddings.

Installation

Install the required packages:

pip install langchain-chroma langchain-openai chromadb

For persistent storage, Chroma uses SQLite by default, included with chromadb. For client-server mode, run a Chroma server:

chroma run --host localhost --port 8000

For persistent storage, specify a directory:

vector_store = Chroma(
    collection_name="langchain_example",
    embedding_function=embedding_function,
    persist_directory="./chroma_db"
)

For detailed installation guidance, see Chroma Integration.

Configuration Options

Customize the Chroma vector store during initialization:

  • embedding_function: Embedding function for dense vectors.
  • collection_name: Name of the Chroma collection (default: langchain).
  • persist_directory: Directory for persistent storage (default: None for in-memory).
  • client: A chromadb.Client instance for custom configurations (e.g., client-server mode).
  • collection_metadata: Metadata for the collection (e.g., {"hnsw:space": "cosine"}).
  • distance_strategy: Distance metric (COSINE, L2, IP; default: COSINE).

Example with persistent storage and cosine distance:

import chromadb
client = chromadb.PersistentClient(path="./chroma_db")
vector_store = Chroma(
    collection_name="langchain_example",
    embedding_function=embedding_function,
    client=client,
    collection_metadata={"hnsw:space": "cosine"}
)

Core Features

1. Indexing Documents

Indexing is the foundation of similarity search, enabling Chroma to store and organize embeddings for rapid retrieval. The Chroma vector store supports indexing raw texts, pre-computed embeddings, and documents with metadata, offering flexibility for various use cases.

  • Key Methods:
    • from_documents(documents, embedding, collection_name="langchain", persist_directory=None, client=None, collection_metadata=None, **kwargs): Creates a vector store from a list of Document objects.
      • Parameters:
        • documents: List of Document objects with page_content and optional metadata.
        • embedding: Embedding function for dense vectors.
        • collection_name: Name of the collection.
        • persist_directory: Directory for persistent storage.
        • client: Chroma client instance.
        • collection_metadata: Metadata for the collection (e.g., distance metric).
      • Returns: A Chroma instance.
    • from_texts(texts, embedding, metadatas=None, ids=None, collection_name="langchain", persist_directory=None, client=None, collection_metadata=None, **kwargs): Creates a vector store from a list of texts.
    • add_documents(documents, ids=None, **kwargs): Adds documents to an existing collection.
      • Parameters:
        • documents: List of Document objects.
        • ids: Optional list of unique IDs.
      • Returns: List of assigned IDs.
    • add_texts(texts, metadatas=None, ids=None, **kwargs): Adds texts to an existing collection.
  • Index Types:

Chroma uses HNSW (Hierarchical Navigable Small World) indexing by default for approximate nearest-neighbor search, optimized for speed and scalability. The index is configured via collection_metadata:


  • hnsw:space: Distance metric (cosine, l2, ip).
  • hnsw:M: Maximum number of neighbor connections (default: 16).
  • hnsw:ef_construction: Size of the dynamic list during index construction (default: 200).
  • Example:
  • vector_store = Chroma(
        collection_name="langchain_example",
        embedding_function=embedding_function,
        collection_metadata={"hnsw:space": "cosine", "hnsw:M": 32, "hnsw:ef_construction": 100}
    )
  • Example (Dense Indexing):
  • from langchain_core.documents import Document
  documents = [
      Document(page_content="The sky is blue.", metadata={"source": "sky", "id": 1}),
      Document(page_content="The grass is green.", metadata={"source": "grass", "id": 2}),
      Document(page_content="The sun is bright.", metadata={"source": "sun", "id": 3})
  ]
  vector_store = Chroma.from_documents(
      documents,
      embedding=embedding_function,
      collection_name="langchain_example",
      persist_directory="./chroma_db"
  )
  • Example (Custom IDs):
  • vector_store.add_texts(
      texts=["The sky is blue."],
      metadatas=[{"source": "sky"}],
      ids=["doc1"]
  )
  • Collection Management:
    • Chroma creates collections automatically if they don’t exist.
    • Use reset_collection() to clear a collection:
    • vector_store.reset_collection()

For advanced indexing, see Document Indexing.

Similarity search retrieves documents closest to a query based on vector similarity, powering applications like semantic search and question answering.

  • Key Methods:
    • similarity_search(query, k=4, filter=None, **kwargs): Searches for the top k documents using vector similarity.
      • Parameters:
        • query: Input text.
        • k: Number of results (default: 4).
        • filter: Optional metadata filter dictionary.
        • kwargs: Additional query parameters (e.g., include for metadata).
      • Returns: List of Document objects.
    • similarity_search_with_score(query, k=4, filter=None, **kwargs): Returns tuples of (Document, score), where scores depend on the distance metric.
    • similarity_search_by_vector(embedding, k=4, filter=None, **kwargs): Searches using a pre-computed embedding.
    • max_marginal_relevance_search(query, k=4, fetch_k=20, lambda_mult=0.5, filter=None, **kwargs): Uses Maximal Marginal Relevance (MMR) to balance relevance and diversity.
      • Parameters:
        • fetch_k: Number of candidates to fetch (default: 20).
        • lambda_mult: Diversity weight (0 for max diversity, 1 for min; default: 0.5).
  • Distance Metrics:
    • COSINE: Cosine similarity, ideal for normalized embeddings (default).
    • L2: Euclidean distance, measuring straight-line distance.
    • IP: Inner product, suited for unnormalized embeddings.
    • Set via collection_metadata:
    • vector_store = Chroma(
        collection_name="langchain_example",
        embedding_function=embedding_function,
        collection_metadata={"hnsw:space": "l2"}
    )
  • Example (Vector Similarity Search):
  • query = "What is blue?"
  results = vector_store.similarity_search_with_score(
      query,
      k=2,
      filter={"source": "sky"}
  )
  for doc, score in results:
      print(f"Text: {doc.page_content}, Metadata: {doc.metadata}, Score: {score}")
  • Example (MMR Search):
  • results = vector_store.max_marginal_relevance_search(
      query,
      k=2,
      fetch_k=10,
      filter={"source": {"$eq": "sky"}}
  )
  for doc in results:
      print(f"MMR Text: {doc.page_content}, Metadata: {doc.metadata}")
  • Search Parameters:
    • Use include in kwargs to specify returned fields (e.g., ["metadatas", "documents"]).
    • Example:
    • results = vector_store.similarity_search(
        query,
        k=2,
        include=["metadatas", "distances"]
    )

For querying strategies, see Querying Vector Stores.

3. Metadata Filtering

Metadata filtering refines search results using key-value conditions, supporting exact matches and basic operators.

  • Filter Syntax:
    • Filters are dictionaries with metadata keys and values, supporting $eq, $ne, $gt, $gte, $lt, $lte, $in, $nin.
    • Example:
    • filter = {
        "$and": [
            {"source": {"$eq": "sky"}},
            {"id": {"$gt": 0}}
        ]
    }
    results = vector_store.similarity_search(query, k=2, filter=filter)
  • Advanced Filtering:
    • Supports $and and $or for logical combinations.
    • Example:
    • filter = {
        "$or": [
            {"source": {"$eq": "sky"}},
            {"source": {"$eq": "grass"}}
        ]
    }
    results = vector_store.similarity_search(query, k=2, filter=filter)

For advanced filtering, see Metadata Filtering.

4. Persistence and Serialization

Chroma supports persistent storage, with options for in-memory or on-disk collections.

  • Key Methods:
    • from_texts(texts, embedding, metadatas=None, ids=None, collection_name="langchain", persist_directory=None, client=None, **kwargs): Creates a new collection or adds to an existing one.
    • persist(): Saves the collection to disk (required for in-memory instances with persist_directory).
    • delete(ids=None, where=None, **kwargs): Deletes documents by IDs or metadata filter.
      • Parameters:
        • ids: List of document IDs.
        • where: Metadata filter dictionary.
    • reset_collection(): Deletes all documents in the collection.
  • Example:
  • vector_store = Chroma.from_texts(
      texts=["The sky is blue."],
      embedding=embedding_function,
      collection_name="langchain_example",
      persist_directory="./chroma_db"
  )
  vector_store.persist()
  vector_store.delete(where={"source": "sky"})
  • Storage Modes:
    • In-Memory: Data is lost when the process ends (default if persist_directory is None).
    • Persistent: Data is saved to disk using SQLite (specify persist_directory).
    • Client-Server: Connect to a Chroma server for distributed storage:
    • client = chromadb.HttpClient(host="localhost", port=8000)
    vector_store = Chroma(
        collection_name="langchain_example",
        embedding_function=embedding_function,
        client=client
    )

5. Document Store Management

Chroma stores documents as records with embeddings, metadata, and IDs.

  • Record Structure:
    • Each record includes:
      • id: Unique identifier (auto-generated or user-specified).
      • embedding: Dense vector.
      • metadata: Dictionary with custom fields (e.g., source, id).
      • document: Text content.
    • Example Record:
    • {
      "id": "doc1",
      "embedding": [0.1, 0.2, ...],
      "metadata": {"source": "sky", "id": 1},
      "document": "The sky is blue."
    }
  • Custom IDs:
    • Specify ids to control document identifiers, ensuring uniqueness.
    • Example:
    • vector_store.add_texts(
        texts=["The sky is blue."],
        metadatas=[{"source": "sky"}],
        ids=["doc1"]
    )
  • Example:
  • documents = [
      Document(page_content="The sky is blue.", metadata={"source": "sky", "id": 1})
  ]
  vector_store.add_documents(documents, ids=["doc1"])

Performance Optimization

Chroma is designed for speed and simplicity, but performance depends on configuration.

Index Configuration

  • HNSW Parameters:
    • M: Maximum neighbor connections (higher for better accuracy, lower for speed).
    • ef_construction: Size of the dynamic list during indexing (higher for better quality, slower indexing).
    • Example:
    • vector_store = Chroma(
        collection_name="langchain_example",
        embedding_function=embedding_function,
        collection_metadata={"hnsw:M": 32, "hnsw:ef_construction": 100}
    )
  • Search Parameters:
    • Adjust ef (search-time dynamic list size) for speed vs. accuracy:
    • results = vector_store.similarity_search(
        query,
        k=2,
        ef=100
    )

Batch Processing

  • Use batch_size in add_texts or add_documents to optimize indexing:
  • vector_store.add_texts(
      texts=["The sky is blue.", "The grass is green."],
      batch_size=500
  )

Persistent Storage

  • Use persist_directory to reduce memory usage for large datasets:
  • vector_store = Chroma(
      collection_name="langchain_example",
      embedding_function=embedding_function,
      persist_directory="./chroma_db"
  )

For optimization tips, see Vector Store Performance and Chroma Documentation.

Practical Applications

Chroma powers diverse AI applications:

  1. Semantic Search:
    • Index documents for natural language queries.
    • Example: A knowledge base for technical manuals.
  1. Question Answering:
  1. Recommendation Systems:
    • Index product descriptions for personalized recommendations.
  1. Chatbot Context:

Try the Document Search Engine Tutorial.

Comprehensive Example

Here’s a complete semantic search system with metadata filtering and MMR:

from langchain_chroma import Chroma
from langchain_openai import OpenAIEmbeddings
from langchain_core.documents import Document

# Initialize embeddings
embedding_function = OpenAIEmbeddings(model="text-embedding-3-large")

# Create documents
documents = [
    Document(page_content="The sky is blue and vast.", metadata={"source": "sky", "id": 1}),
    Document(page_content="The grass is green and lush.", metadata={"source": "grass", "id": 2}),
    Document(page_content="The sun is bright and warm.", metadata={"source": "sun", "id": 3})
]

# Initialize vector store
vector_store = Chroma.from_documents(
    documents,
    embedding=embedding_function,
    collection_name="langchain_example",
    persist_directory="./chroma_db",
    collection_metadata={"hnsw:space": "cosine"}
)

# Similarity search
query = "What is blue?"
results = vector_store.similarity_search_with_score(
    query,
    k=2,
    filter={"source": {"$eq": "sky"}}
)
for doc, score in results:
    print(f"Text: {doc.page_content}, Metadata: {doc.metadata}, Score: {score}")

# MMR search
mmr_results = vector_store.max_marginal_relevance_search(
    query,
    k=2,
    fetch_k=10
)
for doc in mmr_results:
    print(f"MMR Text: {doc.page_content}, Metadata: {doc.metadata}")

# Delete documents
vector_store.delete(where={"source": "sky"})
vector_store.persist()

Output:

Text: The sky is blue and vast., Metadata: {'source': 'sky', 'id': 1}, Score: 0.1234
MMR Text: The sky is blue and vast., Metadata: {'source': 'sky', 'id': 1}
MMR Text: The sun is bright and warm., Metadata: {'source': 'sun', 'id': 3}

Error Handling

Common issues include:

  • Dimension Mismatch: Ensure embedding dimensions match the collection configuration.
  • Empty Collection: Check if data is indexed before querying.
  • Persistence Issues: Verify persist_directory is writable for persistent storage.
  • Invalid Filter: Check filter syntax for correct operators and types.

See Troubleshooting.

Limitations

  • No Hybrid Search: Chroma lacks built-in support for combining vector and keyword search.
  • Filter Complexity: Metadata filters are less expressive than Weaviate or Pinecone.
  • Client-Server Setup: Requires manual server configuration for distributed use.
  • In-Memory Default: Data is lost without explicit persistence.

Conclusion

LangChain’s Chroma vector store is a lightweight, powerful solution for similarity search, combining Chroma’s simplicity with LangChain’s ease of use. Its support for HNSW indexing, metadata filtering, and persistent storage makes it ideal for semantic search, question answering, and recommendation systems. Start experimenting with Chroma to build intelligent, scalable AI applications.

For official documentation, visit LangChain Chroma.