What is hidden partitioning?

Hidden partitioning is Iceberg's approach to partitioning where the partition values are derived automatically from a table's actual data columns, without exposing separate, artificial partition columns that users have to know about and explicitly filter on in their queries.

CREATE TABLE events (
  id BIGINT,
  event_time TIMESTAMP,
  data STRING
)
PARTITIONED BY (day(event_time));

-- Users query the natural column directly
SELECT * FROM events WHERE event_time >= '2026-01-15';

In a traditional Hive-style table, a user typically needs to know a table is partitioned by, say, a separate event_date column and explicitly filter on it to get efficient pruning — forgetting that filter, or filtering only on the underlying timestamp instead, silently results in a full table scan; with hidden partitioning, Iceberg automatically translates a filter on the natural column (event_time above) into the correct partition pruning behind the scenes.

This eliminates what's sometimes called the "missing WHERE clause" foot-gun common in Hive tables: because Iceberg's query planner always applies partition pruning based on its own tracked partition spec rather than relying on the user writing partition-aware predicates, it's structurally much harder to accidentally trigger an expensive full table scan just by phrasing a filter slightly differently than expected.