AI / LlamaIndex Interview Questions
How do you evaluate a LlamaIndex RAG pipeline for faithfulness?
Faithfulness measures whether a generated answer is actually supported by the retrieved source Nodes, rather than the LLM adding unsupported claims or hallucinating details not present in the retrieved context.
LlamaIndex provides FaithfulnessEvaluator for exactly this, using an LLM as a judge: it's given the generated response and its source Nodes, and asked whether every claim in the response is backed by that source text.
from llama_index.core.evaluation import FaithfulnessEvaluator evaluator = FaithfulnessEvaluator(llm=llm) result = evaluator.evaluate_response(response=response) print(result.passing)
It's often run alongside RelevancyEvaluator, which checks whether the response and sources actually address the query, and CorrectnessEvaluator, which compares against a reference answer. Running these across a labeled set of queries with BatchEvalRunner gives an aggregate score you can track as you tune chunk size, similarity_top_k, or the response mode.
More Related questions...
