Web / Apache Lucene Interview questions
How do you troubleshoot OOM errors in a Lucene-based application?
Out-of-memory errors in a Lucene application usually trace back to one of a small set of causes, so a systematic pass through them beats guessing:
- Check the RAM buffer and indexing thread count - a large RAM buffer combined with many concurrent indexing threads multiplies memory pressure at write time.
- Audit DocValues and FieldCache usage - sorting or faceting on high-cardinality fields without proper DocValues can force large in-memory structures to be built.
- Count open IndexReaders/IndexSearchers - forgetting to close old readers after an NRT reopen leaks their in-memory structures until garbage collected, or never, if a live reference lingers.
- Separate heap from off-heap usage - remember MMapDirectory's memory shows up as OS page cache, not JVM heap, so an OOM is a heap-specific problem, not a general "too much memory" problem; don't chase MMapDirectory's resident size as the culprit.
- Review query complexity - queries generating enormous term expansions (a very broad wildcard or fuzzy query) can build large in-memory structures during scoring.
Heap dump analysis narrowing down exactly which objects dominate retained memory is usually faster than reasoning about it in the abstract, since more than one of these causes can be present simultaneously.
More Related questions...