BigData / Apache Iceberg Interview questions
What are deletion vectors, and how do they improve on positional delete files?
Deletion vectors are a more efficient representation for tracking row-level deletes than the original positional delete file approach — rather than a full Avro (or Parquet) file per set of deletes with per-row entries, a deletion vector uses a compact, bitmap-like binary structure to record which row positions in a given data file are deleted.
| Positional Delete Files | Deletion Vectors |
| Structured file format with a row per deleted position. | Compact bitmap-style binary encoding of deleted positions. |
| More storage and processing overhead per delete operation. | Significantly smaller footprint and faster to apply. |
| Can accumulate many small delete files over time. | Designed to consolidate delete information more compactly per file. |
Because a deletion vector is essentially a dense bitmap rather than a general-purpose structured file format, checking whether a given row is deleted becomes a very fast, direct bit lookup rather than requiring the reader to scan through delete file records looking for a match, which meaningfully reduces the CPU and memory overhead merge-on-read tables have historically incurred at query time, especially for tables with a high volume of row-level updates and deletes.
More Related questions...
