BigData / Apache Parquet Interview Questions
What is Z-ordering (Z-order clustering) and how does it help Parquet queries?
Z-ordering is a multi-dimensional data-skipping technique that physically co-locates related rows across multiple filter columns within Parquet files. It maps multiple column values to a single Z-order curve value and sorts data along that curve.
Problem it solves: Standard partitioning and sorting works well for one column. When queries filter on two or more independent columns (e.g., WHERE region='EU' AND product_category='Electronics'), traditional single-column sorting cannot co-locate all relevant rows.
Z-ordering interleaves the bits of the filter column values so that rows with similar values in multiple columns are physically adjacent.
In Delta Lake (Databricks):
OPTIMIZE events ZORDER BY (region, product_category);
After Z-ordering, the same query skips far more row groups because the min/max ranges for both columns are tight within each file.
Trade-off: Z-order OPTIMIZE is a full rewrite of affected files and should be scheduled periodically, not on every write.
More Related questions...
