Prev Next

Web / Apache Solr Interview questions

Why should you use docValues for sorting and faceting fields?

docValues store field values in a column-oriented, on-disk format built at index time, essentially the inverse of the inverted index: instead of term-to-documents, it's document-to-value, optimized for exactly the access pattern sorting, faceting, and grouping need.

Without docValues, Solr falls back to Lucene's uninverted field cache: the first query that sorts or facets on that field triggers Solr to scan the entire index and build the term-to-document mapping in JVM heap, on demand. This has two serious downsides:

  • Heap pressure - the field cache lives entirely in JVM heap and grows with cardinality and document count, a common cause of OutOfMemory errors on large collections.
  • Cold-start latency spikes - the first query (or first query after a commit invalidates it) pays the full build cost, which can be seconds on a large index, causing unpredictable tail latency.

docValues sidesteps both: the structure is built once at index time, stored off-heap using memory-mapped files, and shared cheaply across queries and even across JVM restarts without rebuilding. The trade-off is a modest increase in index size on disk and slightly slower indexing, which is almost always worth it for any field used in sort, facet, or group parameters on non-trivial data volumes - it's considered close to mandatory in current Solr deployments for those use cases.

What is a major downside of relying on the uninverted field cache instead of docValues?
When is the docValues structure for a field built?

More Related questions...

What is Apache Solr? What are the types of fields in Solr? What is a Solr core? What is a Solr collection? What is SolrCloud? What is the purpose of solrconfig.xml? What is the purpose of managed-schema in Solr? Define analyzer in Solr? Describe tokenizer in Solr? What is a filter in the Solr analysis chain? What is an inverted index in Solr? How do you connect to Solr using SolrJ? List the ways to import data into Solr? What is the DataImportHandler in Solr? What are the types of Solr request handlers? How do you apply faceting in a Solr query? What is highlighting in Solr search results? What is the purpose of a query parser in Solr? What is the difference between stored and indexed fields in Solr? What is the difference between the standard query parser and DisMax? What is the difference between DisMax and eDisMax query parsers? What is the difference between q and fq parameters in Solr? What is the difference between soft commit and hard commit? Why is SolrCloud preferred over standalone Solr for production? How does Solr replication work in standalone (master-slave) mode? How does ZooKeeper coordinate a SolrCloud cluster? Why is leader election necessary in SolrCloud shards? What is the difference between NRT, TLOG, and PULL replicas in SolrCloud? When would you choose sharding over replication in SolrCloud? How does Solr handle distributed search across shards? Explain the execution flow of a Solr search request? Explain the lifecycle of a document from indexing to searchability in Solr? Why doesn't Solr return newly indexed documents without a commit? How do you troubleshoot slow Solr queries? How can you optimize indexing throughput in Solr? Explain the internal working of Solr's caching layers? How do you optimize faceted search performance on large collections? What happens when a Solr shard leader goes down? Which is better for autocomplete, EdgeNGram or the Suggester component, and why? How does Solr handle deep pagination, and why is cursorMark recommended over start/rows? Explain the execution flow of an atomic update in Solr? Why should you use docValues for sorting and faceting fields? How do you troubleshoot OutOfMemory errors in a Solr node? What is the difference between Solr's standard scoring and function queries? How does SolrCloud handle a split-brain or network partition scenario? Explain the internal working of Lucene segment merging in Solr? When should you use a custom similarity or scoring function in Solr? How do you optimize Solr for high-throughput near real-time indexing? What is the difference between Apache Solr and Elasticsearch? How do you secure a SolrCloud cluster in production?
Show more question and Answers...


Comments & Discussions