AI / LlamaIndex Interview Questions
1. What is LlamaIndex?
LlamaIndex is an open-source data framework in Python (and TypeScript) built for connecting large language models to your own data, most commonly to power retrieval-augmented generation (RAG) applications. It handles the parts of a RAG pipeline that are easy to get wrong by hand: loading data fro...
2. What is the purpose of LlamaIndex in a RAG pipeline?
In a RAG pipeline, LlamaIndex's job is to bridge the gap between raw, unstructured data and an LLM that can only reason over text placed directly in its prompt. It does this in three stages. First, ingestion : loading documents and splitting them into smaller Nodes. Second, indexing : embedding t...
3. What are Documents and Nodes in LlamaIndex?
A Document is LlamaIndex's container for a raw piece of source data, such as an entire PDF, a web page, or a database row, along with any metadata attached to it like a filename or author. A Node is a smaller chunk produced by splitting a Document, and it's the actual unit that gets embedded, sto...
4. What is a VectorStoreIndex?
A VectorStoreIndex is the most commonly used index type in LlamaIndex. It embeds each Node into a numerical vector using an embedding model, then stores those vectors in a vector store so similar pieces of text can be found through similarity search. When you build one with VectorStoreIndex.from_...
5. What are the types of indices in LlamaIndex?
LlamaIndex ships several index types, each organizing Nodes differently depending on how you plan to query them. VectorStoreIndex - embeds Nodes and retrieves by similarity search; the default choice for most RAG use cases. SummaryIndex (formerly ListIndex) - stores Nodes as a simple sequential l...
6. What is a query engine in LlamaIndex?
A query engine is the object you actually call to ask a question against an index. It's created by calling index.as_query_engine() and packages together a retriever and a response synthesizer into a single, easy-to-use interface. When you call query_engine.query("your question") , it retrieves th...
7. What is a retriever in LlamaIndex?
A retriever is the component responsible for fetching the Nodes most relevant to a given query from an index, without doing anything with an LLM to generate a final answer. For a VectorStoreIndex, the default retriever is a VectorIndexRetriever , which embeds the query and performs a similarity s...
8. What is a response synthesizer?
A response synthesizer is the piece of a query engine that takes the Nodes returned by a retriever, combines them with the user's query, and prompts the LLM to produce the final answer text. It's separate from the retriever on purpose: retrieval decides which text is relevant, while synthesis dec...
9. What is Settings in LlamaIndex?
Settings is LlamaIndex's global configuration object, used to set defaults such as the LLM ( Settings.llm ), the embedding model ( Settings.embed_model ), the chunk size, and the node parser used across an application. It replaced the older ServiceContext object, which had to be explicitly create...
10. What is a node parser or text splitter in LlamaIndex?
A node parser, also called a text splitter, is the component that breaks a Document's text into smaller Node chunks before embedding and indexing. The default is SentenceSplitter , which tries to split on sentence boundaries while respecting a target chunk size and overlap so that context isn't h...
11. How do you use SimpleDirectoryReader?
SimpleDirectoryReader is LlamaIndex's default loader for pulling local files into Documents. Point it at a folder and it will read the files inside, inferring the right parser for common formats like .txt , .pdf , .docx , .csv , and .md automatically. from llama_index.core import SimpleDirectoryR...
12. What is LlamaHub?
LlamaHub is LlamaIndex's registry of community and first-party integrations: data loaders/readers for sources beyond local files (Notion, Slack, Google Drive, SQL databases, web pages, and more), plus tools and pre-built LlamaPacks. Rather than every project reimplementing a loader for, say, a Co...
13. Describe the ingestion pipeline in LlamaIndex?
The IngestionPipeline is LlamaIndex's way of chaining together the steps that turn raw Documents into indexed, embedded Nodes, so those steps don't need to be called manually one by one. You configure it with a list of transformations , typically a node parser followed by an embedding model, and ...
14. What is a chat engine in LlamaIndex?
A chat engine wraps a query engine with conversation memory, so it can answer follow-up questions that depend on earlier turns, which a plain query engine can't do since it treats every call independently. You create one with index.as_chat_engine(chat_mode="...") . Common modes include condense_q...
15. What are the response modes available in LlamaIndex query engines?
The response_mode parameter controls how a response synthesizer turns retrieved Nodes into a final answer, and LlamaIndex offers several depending on how much thoroughness versus speed you need. Mode Behavior refine Processes Nodes one at a time, updating the answer sequentially with one LLM call...
16. What is the difference between VectorStoreIndex and SummaryIndex?
A VectorStoreIndex embeds every Node and answers queries by finding the Nodes whose embeddings are most similar to the query, so it's built for selective, targeted retrieval over potentially large corpora. A SummaryIndex stores Nodes as a plain sequential list and, by default, sends all of them t...
17. How does similarity_top_k affect retrieval?
similarity_top_k sets how many of the closest-matching Nodes a vector retriever returns for a given query. A higher value returns more candidate Nodes; a lower value returns fewer. Raising it tends to improve recall , the chance that the actually relevant Node is somewhere in the retrieved set, e...
18. What is the difference between a query engine and a chat engine?
A query engine is stateless: each call to .query() is handled independently, with no memory of previous questions. It's built for single-turn, programmatic Q&A, such as answering one lookup at a time from an API. A chat engine is stateful: it holds a memory buffer of the conversation and can reso...
19. How do node postprocessors work in LlamaIndex?
Node postprocessors run after retrieval but before synthesis, taking the list of retrieved Nodes and filtering, reordering, or re-scoring them before they reach the LLM. Common examples include SimilarityPostprocessor , which drops Nodes below a similarity score cutoff; KeywordNodePostprocessor ,...
20. Why should you use metadata filtering in retrieval?
Metadata filtering lets you narrow a similarity search to only the Nodes whose metadata matches specific conditions, such as a date range, a source document, or a tenant ID, before or alongside the embedding comparison. This matters because vector similarity alone doesn't understand hard constrai...
21. What is the difference between refine and compact response modes?
Both refine and compact build an answer by working through the retrieved Nodes with the LLM, but they differ in how many calls that takes. refine processes Nodes one at a time: it generates an initial answer from the first Node, then feeds each subsequent Node to the LLM along with the current an...
22. How does the SubQuestionQueryEngine work?
SubQuestionQueryEngine is designed for complex questions that really require pulling information from multiple sources or multiple angles of the same source, something a single retrieval pass often can't answer well. Given a query, it first uses an LLM to break the question into smaller sub-quest...
23. What is a RouterQueryEngine and when would you use it?
A RouterQueryEngine picks which of several query engine tools should handle a given query, using an LLM-based selector that reads each tool's description and decides the best fit, rather than sending every query to every tool. You'd use it when you have distinct data sources or index types servin...
24. Why is chunk size important in LlamaIndex?
Chunk size determines how large each Node is when a Document is split, and it directly shapes both retrieval quality and cost, so getting it wrong in either direction hurts the pipeline. Chunks that are too large tend to mix multiple topics together, which dilutes the embedding, making it a fuzzy...
25. How do you persist and reload an index in LlamaIndex?
Building an index from scratch every time an application starts is wasteful, since it means re-embedding every document. LlamaIndex avoids that with StorageContext , which can save the docstore, index store, and vector store to disk and load them back later. # Persist index . storage_context . pe...
26. What is the difference between LlamaIndex and LangChain?
LlamaIndex and LangChain both help build LLM applications, but they grew out of different core focuses. LlamaIndex started as a data framework specialized in ingestion, indexing, and retrieval, with deep tooling around chunking strategies, index types, and RAG-specific evaluation. LangChain start...
27. How does the SentenceWindowNodeParser improve retrieval quality?
SentenceWindowNodeParser splits Documents into Nodes at the individual sentence level, but instead of throwing away surrounding context, it stores a window of neighboring sentences in each Node's metadata under a key like window . This matters because embedding a single sentence gives a very prec...
28. When should you use auto-merging retrieval?
Auto-merging retrieval is built for cases where relevant information tends to span more than one adjacent chunk, and you don't want retrieval fragmented across several small, disconnected pieces. It relies on a HierarchicalNodeParser that builds parent-child chunk hierarchies, for example large p...
29. What is HyDE and how does it help retrieval?
HyDE (Hypothetical Document Embeddings) is a query transformation technique where, instead of embedding the user's raw question directly, an LLM first generates a hypothetical answer to that question, and it's the embedding of that generated answer that gets used for similarity search. The idea a...
30. How do you integrate a custom vector store like Pinecone or Chroma with LlamaIndex?
LlamaIndex integrates external vector stores through a thin wrapper class per provider, such as PineconeVectorStore or ChromaVectorStore , which implements a common interface so the rest of the framework doesn't need to know which backend is being used. from llama_index.vector_stores.chroma impor...
31. What is the difference between ReActAgent and FunctionCallingAgent?
Both are ways to build tool-using agents in LlamaIndex, but they rely on different mechanisms for deciding which tool to call. ReActAgent uses the ReAct prompting pattern: the LLM writes out a free-text loop of Thought, Action, and Observation, where the "Action" line names a tool and its argumen...
32. How does LlamaIndex support structured data querying such as SQL?
LlamaIndex can answer natural-language questions against structured databases through query engines like NLSQLTableQueryEngine , which bridges plain English and SQL without the developer hand-writing queries. The engine introspects the target database's schema, table names, column names, and type...
33. Why use CohereRerank or LLMRerank as a node postprocessor?
Raw vector similarity search is a fast, approximate way to find candidate Nodes, but embedding similarity doesn't always line up perfectly with true relevance to a specific query, especially past the top handful of results. A common pattern is to deliberately over-fetch, for example retrieving si...
34. What is the role of the CallbackManager in LlamaIndex?
The CallbackManager is LlamaIndex's hook system for observing what happens inside a pipeline, firing events at points like an LLM call starting or finishing, an embedding call, or a retrieval step, without you having to modify the core pipeline code. It's the mechanism behind common debugging nee...
35. How do you evaluate a LlamaIndex RAG pipeline for faithfulness?
Faithfulness measures whether a generated answer is actually supported by the retrieved source Nodes, rather than the LLM adding unsupported claims or hallucinating details not present in the retrieved context. LlamaIndex provides FaithfulnessEvaluator for exactly this, using an LLM as a judge: i...
36. Explain the execution flow of a query in a VectorStoreIndex-based query engine?
When you call query_engine.query("...") on a VectorStoreIndex-backed engine, several distinct steps run in a fixed order before you get a final answer back. flowchart TD A[User query string] --> B{Query transform?} B -->|optional, e.g. HyDE| C[Transformed query] B -->|none| D[Embed query] C --> D...
37. Explain the internal working of the IngestionPipeline caching mechanism?
The IngestionPipeline 's cache exists to avoid redoing expensive work, mainly embedding calls, when the pipeline is re-run on data it has already processed. Internally, the IngestionCache computes a hash from each input Node's content combined with the specific transformation and its configuratio...
38. Explain the lifecycle of a Node from Document to retrieval?
A Node passes through several distinct stages between when raw data enters LlamaIndex and when it's finally used to answer a query. flowchart LR A[Document loaded by a reader] --> B[Node parser splits into Nodes] B --> C[Relationships assigned: PREVIOUS/NEXT/PARENT/CHILD] C --> D[Optional metadat...
39. What is the difference between PropertyGraphIndex and KnowledgeGraphIndex?
KnowledgeGraphIndex was LlamaIndex's original graph-based index. It uses an LLM to extract simple (subject, predicate, object) triplets from text and stores them in a graph store, with retrieval typically done through keyword matching or basic graph traversal from entities mentioned in the query....
40. How can you optimize token usage and cost in a large-scale LlamaIndex deployment?
Cost in a LlamaIndex application comes mainly from embedding calls and LLM calls, so optimization means reducing unnecessary calls on both fronts without hurting answer quality too much. Cache aggressively. Use the IngestionPipeline's cache so unchanged documents aren't re-embedded, and consider ...
41. Explain the internal working of AgentWorkflow and event-driven workflows in LlamaIndex?
LlamaIndex Workflows are an event-driven orchestration primitive: instead of writing one linear function, you define a set of steps , each a Python method decorated with @step , and each step declares what Event type it consumes and what Event type it produces. flowchart LR A[StartEvent] --> B[St...
42. How do you troubleshoot poor retrieval relevance in a LlamaIndex application?
When a RAG application returns answers that miss the point or cite the wrong context, the fix usually starts by actually inspecting the source_nodes on the response object, rather than guessing, since that tells you whether the problem is retrieval or synthesis. Check what was actually retrieved....
43. What happens internally when you call index.as_query_engine()?
Calling index.as_query_engine() looks like a single simple call, but it assembles several components behind the scenes using sensible defaults so you don't have to wire them up manually. It constructs a retriever appropriate to the index's type, for example a VectorIndexRetriever for a VectorStor...
44. How does LlamaIndex handle asynchronous querying at scale?
Most of the slow parts of a LlamaIndex pipeline, embedding calls, vector store lookups, and LLM calls, are I/O-bound network requests, which makes them a natural fit for Python's asyncio rather than running everything sequentially. LlamaIndex exposes async counterparts throughout the stack: aquer...
45. Explain the difference between the low-level composition API and the high-level API in LlamaIndex?
The high-level API is the one-liner style most tutorials use: VectorStoreIndex.from_documents(documents) followed by index.as_query_engine() . It picks sensible defaults for the node parser, retriever, and synthesizer automatically, which is great for getting started quickly. The low-level compos...
46. Why doesn't increasing similarity_top_k always improve answer quality?
It's tempting to assume retrieving more Nodes can only help, since the correct one is more likely to be included, but in practice a higher similarity_top_k comes with real costs that can offset or even reverse that benefit. First, every additional Node retrieved is rarely perfectly relevant; padd...
47. How do you design a hybrid search system combining vector and keyword retrieval in LlamaIndex?
Pure vector similarity search is excellent at capturing semantic meaning but can miss exact matches on rare terms, product codes, names, or acronyms that a keyword search would catch immediately, since an embedding might not weight an exact token match heavily. Hybrid search combines both so neit...
48. Explain the internal working of tree_summarize response synthesis?
tree_summarize is built for queries that genuinely need to draw on many retrieved Nodes holistically, such as "summarize the key risks across all these reports," where sequentially refining through Nodes one by one, as refine does, tends to over-weight whichever Node happened to be processed firs...
49. How would you architect a multi-tenant LlamaIndex application with metadata filtering per tenant?
A multi-tenant RAG application needs to guarantee that one customer's query never surfaces another customer's data, and the architecture choice comes down to how strict that isolation needs to be versus how much operational overhead you can take on. Shared index, metadata-scoped. Every Node is ta...
50. Which is better and why: sentence-window retrieval vs auto-merging retrieval for long documents?
Neither is universally better; the right choice depends on whether a typical query in your application needs a single pinpoint fact or a broader coherent passage, since the two techniques solve different failure modes of fixed-size chunking. Sentence-window retrieval embeds individual sentences f...
