BigData / Apache Iceberg Interview questions
Explain the lifecycle of a write operation (commit) in Apache Iceberg?
A write to an Iceberg table moves through a defined sequence: preparing new data (and, for merge-on-read, delete) files, building new manifest and manifest list metadata describing the resulting state, and finally an atomic commit that swaps the catalog's pointer to a brand-new metadata file.
Everything up through building the new manifest list and metadata file happens without touching the catalog at all — it's only the final step, the atomic pointer swap, that actually makes the new state visible to other readers and writers, which is exactly what gives Iceberg its all-or-nothing commit guarantee: any failure before that final atomic swap simply leaves the table's current state completely untouched.
This design also means a failed or aborted write leaves no partial trace in the table's visible state: orphaned data files from an incomplete write might exist in storage, but since no committed snapshot ever references them, they're invisible to any query and can be safely cleaned up later by a maintenance operation without affecting the table's correctness.
More Related questions...