BigData / Apache Hudi Interview Questions
How does Hudi's Non-Blocking Concurrency Control work internally?
Non-Blocking Concurrency Control (NBCC), introduced in Hudi 1.0, is designed for the specific case where multiple writers — for example, a streaming ingestion job and an async compaction/clustering job — need to touch the same file group at roughly the same time, something the older OCC-plus-external-lock model handled by simply blocking one of them.
Instead of taking an exclusive lock and forcing one writer to wait or abort, NBCC lets both writers append their changes as separate log blocks within the same file slice, ordering them using event/commit time rather than lock acquisition order. Readers and later table services reconcile these concurrently-written blocks deterministically at merge time, which is what "non-blocking" refers to: neither writer is forced to stall waiting on the other's lock to release. This removes the need for an external lock provider (like Zookeeper) purely to coordinate table-service and ingestion concurrency.
More Related questions...