BigData / Apache Hudi Interview Questions
Explain the internal working of an upsert operation in Hudi?
An upsert moves through three distinct internal stages, and being able to name each one is a strong signal in an interview.
flowchart LR
A[Incoming records] --> B[Index lookup: tag records with target file group]
B --> C{New or existing key?}
C -- New --> D[Route to insert path]
C -- Existing --> E[Route to update path]
D --> F[Write to base file or new log block]
E --> F
F --> G[Commit instant marked completed on timeline]
First, the index tags every incoming record with the file group it belongs to (or marks it as a brand-new insert). Second, records are routed to either the insert or update write path based on that tag. Third, for CoW the affected base file is rewritten with the merged result, while for MoR the change is appended as a new log block; either way, the write only becomes visible to readers once its commit instant is marked completed on the timeline.
The very first internal step of an upsert is:
A write becomes visible to readers only once:
More Related questions...
