BigData / Apache Iceberg Interview questions
Explain how Iceberg handles concurrent writes?
Iceberg uses optimistic concurrency control for writes: multiple writers can prepare their changes (new data files, new manifests) simultaneously without any locking, and conflicts are only detected and resolved at the final commit step, when a writer attempts to atomically swap the catalog's pointer.
Each writer works from a base snapshot it read at the start of its operation; when it attempts to commit, the catalog checks whether that base snapshot is still the current one — if another writer already committed a change in the meantime, the base has moved, and the commit is rejected as a conflict rather than silently applied on top of stale assumptions.
On a conflict, a writer typically retries: it re-reads the new current snapshot, and if its own pending changes are still compatible with that updated state (for example, two writers appending to entirely different partitions rarely genuinely conflict), it reapplies its commit against the new base; genuinely conflicting changes (like two writers both trying to update the exact same rows) require the retry logic to resolve or fail explicitly, since no metadata-only merge can safely resolve a true data-level conflict on its own.
More Related questions...