What embedding models should you use for production RAG systems, and how do you choose between OpenAI and open-source options?

The embedding model is one of the most consequential choices in a RAG system — it determines retrieval quality, cost, latency, and whether data leaves your infrastructure. The right choice depends on your data volume, sensitivity, quality requirements, and deployment environment.

# ── OpenAI embeddings (best quality, external API)
from langchain_openai import OpenAIEmbeddings

oai_embed = OpenAIEmbeddings(
    model='text-embedding-3-small',
    dimensions=512,  # can reduce from 1536 for speed/cost (Matryoshka)
)

# ── Local HuggingFace embeddings (private, free)
from langchain_huggingface import HuggingFaceEmbeddings

hf_embed = HuggingFaceEmbeddings(
    model_name='BAAI/bge-large-en-v1.5',
    model_kwargs={'device': 'cuda'},
    encode_kwargs={'normalize_embeddings': True},
)

# ── Direct sentence-transformers usage
from sentence_transformers import SentenceTransformer

model = SentenceTransformer('BAAI/bge-small-en-v1.5', device='cuda')
texts = ['Hello world', 'Machine learning']
embeds = model.encode(texts, batch_size=64, normalize_embeddings=True)
print(embeds.shape)  # (2, 384)

# ── Benchmark retrieval quality on your own data before committing
# BEIR benchmark: standardised RAG retrieval evaluation
# https://huggingface.co/spaces/mteb/leaderboard — MTEB leaderboard

# Quick retrieval quality check
query   = 'What is machine learning?'
corpus  = ['ML is a type of AI', 'The sky is blue', 'Neural networks learn from data']
q_embed = model.encode(query, normalize_embeddings=True)
c_embed = model.encode(corpus, normalize_embeddings=True)
scores  = c_embed @ q_embed
ranked  = sorted(zip(scores, corpus), reverse=True)
print(ranked)