What is partition evolution, and how does it work internally?

Partition evolution is the ability to change an Iceberg table's partitioning strategy going forward — for example, switching from monthly to daily partitioning as data volume grows — without rewriting any of the table's existing data files, since only new data written after the change uses the new partition spec.

ALTER TABLE events ADD PARTITION FIELD day(event_time);
-- existing data stays partitioned by month; only new writes use the day-level spec

Internally, Iceberg tracks partition specs by version, and both the old and new specs coexist in the table's metadata; when the query planner evaluates a filter, it reads all relevant manifests, groups them by which partition spec each one was written under, and applies the correct partition pruning logic separately for each group — old manifests pruned against month-level bounds, new manifests pruned against day-level bounds — before merging both result sets into one final answer.

This is a meaningfully different capability from Hive-style or Delta Lake tables, both of which historically required a full table rewrite to change partitioning; Iceberg's ability to evolve the partition spec purely as a metadata change, leaving historical data files untouched, is one of the format's most distinctive and frequently cited advantages for tables that need to adapt their partitioning strategy as data volume or query patterns change over a table's lifetime.