When would you choose multi-vector (ColBERT-style) embeddings over single-vector embeddings?

A traditional single-vector embedding compresses an entire piece of text (a document, a passage) down into one fixed-length vector, which necessarily loses some fine-grained, token-level information in the process. Multi-vector embeddings, produced by models like ColBERT, ColPali, or ColQwen, instead represent an object (or query) as a set of multiple vectors, often one per token or region, preserving more granular detail and enabling a more precise "late interaction" comparison between query and document tokens at search time.

This granularity tends to matter most for retrieval tasks where fine-grained, token-level matching genuinely improves relevance beyond what a single pooled vector can capture, dense technical documents, code, or, in the case of vision-language models like ColPali, retrieving directly from document images (like scanned PDFs or slides) without needing a separate text-extraction step first.

The trade-off is cost: storing and searching multiple vectors per object instead of one meaningfully increases both storage footprint and query computation compared to single-vector search, which is why multi-vector embeddings tend to be reached for specifically when benchmarking shows a genuine relevance improvement for a given retrieval task, rather than adopted by default for every use case regardless of whether that improvement materializes.