BigData / Apache Hudi Interview Questions
How does Hudi ensure ACID guarantees on cloud object storage?
Cloud object stores like S3 don't natively offer multi-file atomic transactions, so Hudi has to build ACID guarantees itself on top of storage primitives it can rely on — mainly atomic single-file writes/renames and the timeline's own ordering.
- Atomicity — a commit's data files are all written first, and only after that succeeds does Hudi atomically write the commit's completed marker on the timeline; readers checking the timeline either see the whole commit or none of it.
- Consistency — schema validation and the timeline's ordered instants ensure the table never exposes a structurally invalid intermediate state.
- Isolation — readers get snapshot isolation by only reading data as of the latest completed instant at query start, unaffected by writers that start afterward; concurrent writers are coordinated via OCC (with external locks) or NBCC.
- Durability — once a commit's marker is completed on the timeline, its data is durably persisted in the underlying object store, which itself guarantees durability once a write succeeds.
More Related questions...