Web / Apache Lucene Interview questions
Explain the internal working of Lucene's MMapDirectory I/O?
MMapDirectory reads index files using the operating system's memory-mapping facility (mmap) instead of issuing explicit read() system calls the way NIOFSDirectory does. The file's contents are mapped directly into the process's virtual address space, and pages are faulted in from disk by the OS on first access.
This has real consequences:
- Random access to index data (typical of search) becomes very cheap after the first touch, since subsequent accesses hit the OS page cache directly with no syscall overhead.
- Memory usage shows up as OS-managed page cache rather than JVM heap, which is why a Lucene process can appear to use far more resident memory than its heap size suggests - that's expected and not a leak.
- On some platforms, unmapping files promptly (especially on older Windows JVMs) has historically had quirks, since a mapped file can't always be deleted or resized while still mapped.
Because it relies on virtual memory rather than heap, MMapDirectory is generally the fastest default Directory on 64-bit systems with adequate free memory for the OS to cache files effectively, which is exactly why it's the default in modern Lucene on most platforms.
More Related questions...