AI / LlamaIndex Interview Questions
Why use CohereRerank or LLMRerank as a node postprocessor?
Raw vector similarity search is a fast, approximate way to find candidate Nodes, but embedding similarity doesn't always line up perfectly with true relevance to a specific query, especially past the top handful of results.
A common pattern is to deliberately over-fetch, for example retrieving similarity_top_k=20 candidates, then apply a reranker like CohereRerank or LLMRerank as a node postprocessor. These re-score each candidate against the query using a more accurate, often cross-encoder or LLM-based, relevance model, which is more expensive per item but far more precise than raw cosine similarity, and then keep only the top few, such as the top 3.
The net effect is retrieval that combines the speed of vector search for casting a wide net with the accuracy of a heavier model for the final cut, which typically improves answer quality more than simply raising similarity_top_k on its own would.
More Related questions...
