Database / ChromaDB Interview Questions
How do you use HuggingFace models as embedding functions in ChromaDB?
ChromaDB provides a HuggingFaceEmbeddingFunction that calls the HuggingFace Inference API (cloud-hosted), and a SentenceTransformerEmbeddingFunction for running any Sentence Transformer model locally. For production use without per-call API costs, local Sentence Transformer models are the more common choice.
import chromadb from chromadb.utils import embedding_functions import os client = chromadb.Client() # Option 1: HuggingFace Inference API (cloud, requires API key) ef_hf_api = embedding_functions.HuggingFaceEmbeddingFunction( api_key=os.environ["HUGGINGFACE_API_KEY"], model_name="sentence-transformers/all-MiniLM-L6-v2", ) # Option 2: Local Sentence Transformers (no API key, runs on your machine) ef_local = embedding_functions.SentenceTransformerEmbeddingFunction( model_name="all-MiniLM-L6-v2", # 384-dim, fast # model_name="all-mpnet-base-v2", # 768-dim, higher quality # model_name="BAAI/bge-large-en-v1.5", # excellent quality device="cpu", # or "cuda" for GPU acceleration ) collection = client.create_collection( name="hf_docs", embedding_function=ef_local, metadata={"hnsw:space": "cosine"}, ) collection.add( documents=[ "Open-source language models are becoming more powerful.", "LLaMA and Mistral are popular open-source LLMs.", ], ids=["h1", "h2"], ) results = collection.query( query_texts=["free LLM models"], n_results=2, ) print(results["documents"]) # Popular local models for RAG models = { "BAAI/bge-small-en-v1.5": "384-dim, excellent quality/speed ratio", "BAAI/bge-large-en-v1.5": "1024-dim, top English quality", "intfloat/e5-base-v2": "768-dim, strong multilingual", "thenlper/gte-large": "1024-dim, great for retrieval", }
Trade-offs: HuggingFace Inference API requires no local GPU but costs money and adds latency. Local Sentence Transformers are free, fast (especially on GPU), run offline, and are privacy-preserving — preferred for sensitive data.
More Related questions...