How do you use HuggingFace models as embedding functions in ChromaDB?

ChromaDB provides a HuggingFaceEmbeddingFunction that calls the HuggingFace Inference API (cloud-hosted), and a SentenceTransformerEmbeddingFunction for running any Sentence Transformer model locally. For production use without per-call API costs, local Sentence Transformer models are the more common choice.

import chromadb
from chromadb.utils import embedding_functions
import os

client = chromadb.Client()

# Option 1: HuggingFace Inference API (cloud, requires API key)
ef_hf_api = embedding_functions.HuggingFaceEmbeddingFunction(
    api_key=os.environ["HUGGINGFACE_API_KEY"],
    model_name="sentence-transformers/all-MiniLM-L6-v2",
)

# Option 2: Local Sentence Transformers (no API key, runs on your machine)
ef_local = embedding_functions.SentenceTransformerEmbeddingFunction(
    model_name="all-MiniLM-L6-v2",    # 384-dim, fast
    # model_name="all-mpnet-base-v2", # 768-dim, higher quality
    # model_name="BAAI/bge-large-en-v1.5",  # excellent quality
    device="cpu",  # or "cuda" for GPU acceleration
)

collection = client.create_collection(
    name="hf_docs",
    embedding_function=ef_local,
    metadata={"hnsw:space": "cosine"},
)

collection.add(
    documents=[
        "Open-source language models are becoming more powerful.",
        "LLaMA and Mistral are popular open-source LLMs.",
    ],
    ids=["h1", "h2"],
)

results = collection.query(
    query_texts=["free LLM models"],
    n_results=2,
)
print(results["documents"])

# Popular local models for RAG
models = {
    "BAAI/bge-small-en-v1.5":  "384-dim, excellent quality/speed ratio",
    "BAAI/bge-large-en-v1.5":  "1024-dim, top English quality",
    "intfloat/e5-base-v2":     "768-dim, strong multilingual",
    "thenlper/gte-large":      "1024-dim, great for retrieval",
}

Trade-offs: HuggingFace Inference API requires no local GPU but costs money and adds latency. Local Sentence Transformers are free, fast (especially on GPU), run offline, and are privacy-preserving — preferred for sensitive data.