What is multi-query retrieval?

Multi-query retrieval addresses a key weakness of single-vector search: a user's question may be phrased in a way that doesn't closely match how the relevant information is worded in the document store. MultiQueryRetriever solves this by using an LLM to automatically generate several alternative phrasings of the query, running each against the vector store, and deduplicating the union of all results.

from langchain.retrievers.multi_query import MultiQueryRetriever
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(temperature=0)

retriever = MultiQueryRetriever.from_llm(
    retriever=vectorstore.as_retriever(),
    llm=llm,
)

# For a query like "What is LangChain memory?"
# The LLM might generate:
#   1. "How does LangChain handle conversation state?"
#   2. "What memory classes are available in LangChain?"
#   3. "How do you persist context between LangChain calls?"
# Then retrieves for all 3 and deduplicates
results = retriever.invoke("What is LangChain memory?")

Multi-query retrieval improves recall — you're less likely to miss relevant documents due to vocabulary mismatch — but it increases latency and cost since it makes multiple LLM calls (for query generation) and multiple vector search calls per user query. It works best for knowledge bases with varied terminology or when users ask high-level questions that could be answered by multiple document sections.

Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!

Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.

Invest now!!! Get Free equity stock (US, UK only)!

Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.

The Robinhood app makes it easy to trade stocks, crypto and more.

Webull! Receive free stock by signing up using the link: Webull signup.