Explain the internal working of Qdrant's segment optimizer/merging?

The segment optimizer is a background process that continuously monitors a collection's segments and reorganizes them — merging small segments into larger ones and reclaiming space from deleted points — to keep query performance from degrading as a collection experiences ongoing writes.

Because each segment maintains its own HNSW index, having many small segments means a search has to query many separate small indexes and merge their results, which carries more per-segment overhead than querying fewer, larger indexes covering the same total data; the optimizer's merging work directly addresses this by consolidating small segments as they accumulate from ongoing writes.

The optimizer also handles reclaiming space from deleted or updated points, which (as with LanceDB and similar systems) aren't erased immediately at delete time but instead marked and later cleaned up; a segment with a high proportion of deleted points is a candidate for vacuuming — rewritten to exclude the deleted data — which is what keeps storage from growing unboundedly under heavy update/delete churn over a collection's lifetime.