Web / Apache Lucene Interview questions
Explain the lifecycle of an IndexWriter commit?
A commit is what makes indexing changes durable and visible to newly opened readers. Understanding the stages helps explain why a commit is more expensive than a plain flush.
sequenceDiagram
participant App
participant IndexWriter
participant Directory
App->>IndexWriter: addDocument()/updateDocument()
IndexWriter->>IndexWriter: buffer in RAM
IndexWriter->>Directory: flush() writes new segment
App->>IndexWriter: commit()
IndexWriter->>Directory: fsync segment files
IndexWriter->>Directory: write new segments_N file
Directory-->>App: commit point durable & visible
- Documents are buffered in memory as they're added.
- A flush writes the buffered documents to disk as a new segment, but this alone isn't durable or guaranteed visible.
- commit() fsyncs all relevant files and writes a new
segments_Nfile pointing to the current set of segments - this is the actual durable commit point. - Only after commit (or via a SearcherManager refresh for NRT) do new readers see the changes.
Because fsync is relatively expensive, applications batch many documents between commits rather than committing after every single add.
What actually makes an IndexWriter's changes durable on disk?
Why do applications avoid committing after every single document?
More Related questions...
