Explain how IVF_PQ indexing works internally?

IVF-PQ is built and used in two distinct phases — an index-build phase that happens once (or periodically, as data grows) and a query phase that runs for every search — and understanding both is what makes the speed/accuracy trade-off concrete rather than abstract.

At build time, IVF partitions the full set of vectors into a configured number of clusters via k-means, storing each cluster's centroid; PQ then compresses every vector by splitting it into sub-vectors and separately quantizing each sub-vector against a small codebook, which shrinks the storage footprint substantially.

At query time, the search first compares the query vector only against the (much smaller) set of cluster centroids to pick the nprobe most promising clusters, then computes fast, approximate distances using the compressed PQ codes only within those selected clusters — skipping the vast majority of the dataset entirely, which is the core mechanism behind IVF-PQ's speed advantage over brute-force search.