BigData / Apache Hudi Interview Questions
Explain the execution flow of a Merge-on-Read compaction?
Compaction on a MoR table follows a two-phase schedule-then-execute flow, which is deliberately split so scheduling can be cheap and frequent while the heavier execution work can run separately.
sequenceDiagram
participant W as Writer / Streamer
participant T as Timeline
participant Comp as Compaction Executor
W->>T: Schedule compaction (requested instant)
Note over T: Freezes the set of file slices to compact
Comp->>T: Pick up requested compaction instant
Comp->>Comp: Read base file + all log blocks for each file slice
Comp->>Comp: Merge log blocks into base file (apply payload logic)
Comp->>T: Write new base file version, mark instant completed
Scheduling freezes exactly which file slices and log blocks are included, so ingestion can keep writing new log blocks after that point without interfering with the in-progress compaction. Execution then reads each frozen file slice's base file plus its log blocks, applies the payload class's merge logic record by record, and writes out a new, fully-merged base file version once done.
Why is compaction split into a schedule phase and an execute phase?
During execution, compaction merges log blocks using:
More Related questions...
