Web / Apache Lucene Interview questions
What is an Analyzer in Lucene?
An Analyzer is the pipeline that turns raw text into the stream of terms Lucene actually indexes. It's applied both when documents are indexed and, typically, when queries are parsed - so search terms line up with indexed terms.
Internally, an Analyzer chains together exactly one Tokenizer (splits text into raw tokens) with zero or more TokenFilters (lowercasing, stopword removal, stemming, and so on). Swapping the Analyzer changes how text is broken down without touching any other part of the indexing code.
Choosing the wrong Analyzer is one of the most common causes of "why doesn't my search find this obvious match" bugs, since a mismatch between how content was indexed and how the query was analyzed silently breaks matching.
More Related questions...