What is the purpose of Apache Iceberg?

Apache Iceberg exists to bring reliable, database-like semantics — ACID transactions, consistent snapshots, safe schema and partition changes — to data stored as plain files in a data lake, solving problems that plagued earlier approaches like Hive tables, which tracked table state through fragile, directory-listing-based conventions.

Before formats like Iceberg existed, changing a Hive table's partitioning scheme typically meant rewriting the entire table, adding a column could silently break downstream queries that relied on column position rather than name, and concurrent writers could produce inconsistent results because there was no atomic way to swap a table from one consistent state to another.

Iceberg's purpose is closing that gap specifically: it lets teams build a "lakehouse" — a data lake with warehouse-like reliability — where multiple engines can safely read and write the same tables concurrently, schemas and partitioning can evolve without disruptive full-table rewrites, and every query sees a consistent, point-in-time view of the data rather than a partially-written, inconsistent state.