Web / Apache Lucene Interview questions
What is a TokenFilter in Lucene?
A TokenFilter takes the token stream produced by a Tokenizer (or a previous filter) and transforms it further. Filters are chained one after another, and order matters - each one sees only the output of the one before it.
Common examples include:
- LowerCaseFilter - normalizes case so "Lucene" and "lucene" match.
- StopFilter - removes common low-value words like "the" or "and".
- PorterStemFilter - reduces words to a root form ("running" to "run").
- SynonymGraphFilter - injects equivalent terms at the same position.
A typical English Analyzer chain looks like StandardTokenizer to LowerCaseFilter to StopFilter to PorterStemFilter - each stage narrowing the vocabulary toward what's actually meaningful for matching.
TokenFilters operate on:
Which filter reduces "running" to a root form like "run"?
More Related questions...
