Explain the internal working of Lucene's BM25Similarity?

BM25Similarity scores a term match for a document using three interacting components combined multiplicatively, rather than the simpler additive TF-IDF formula it replaced as Lucene's default.

1. IDF (inverse document frequency) - terms appearing in fewer documents across the corpus get a higher weight, since they're more discriminating.
2. Term frequency saturation - controlled by k1 (default 1.2), this makes additional occurrences of a term within a document contribute less and less to the score, rather than growing linearly, since a document isn't necessarily 10x as relevant just because a term appears 10x more.
3. Length normalization - controlled by b (default 0.75), this compares a document's field length to the average field length across the corpus, so shorter documents matching a term aren't unfairly penalized relative to longer ones, and vice versa.

Both k1 and b are configurable per-field via a custom Similarity, which is how teams tune BM25 for corpora with very different length distributions - product titles versus full articles, for example - without switching away from BM25 entirely.