AI / LlamaIndex Interview Questions
How does LlamaIndex handle asynchronous querying at scale?
Most of the slow parts of a LlamaIndex pipeline, embedding calls, vector store lookups, and LLM calls, are I/O-bound network requests, which makes them a natural fit for Python's asyncio rather than running everything sequentially.
LlamaIndex exposes async counterparts throughout the stack: aquery() and achat() on query and chat engines, aretrieve() on retrievers, and asynthesize() on response synthesizers. Awaiting these lets the event loop overlap network waits instead of blocking on each call one at a time.
import asyncio async def run_all(query_engine, questions): tasks = [query_engine.aquery(q) for q in questions] return await asyncio.gather(*tasks) results = asyncio.run(run_all(query_engine, questions))
For batch workloads, such as answering hundreds of questions or ingesting thousands of documents, wrapping calls in asyncio.gather like this can cut wall-clock time dramatically compared to a synchronous loop, though in production it's typically paired with a semaphore or similar concurrency limit to stay within the LLM or embedding provider's rate limits rather than firing every call at once.
More Related questions...
