BigData / Apache Hudi Interview Questions
Explain the internal working of partial updates in Hudi 1.x?
Traditionally, an update record supplied to Hudi needed to contain the record's full set of columns, even if only one field actually changed — the payload class would merge the "new" full record against the "old" full record. That's wasteful for CDC sources that only emit the columns that actually changed.
Partial updates in Hudi 1.x let an incoming record specify only a subset of columns, with the merge engine reading the existing record's other column values from storage and combining them with just the supplied fields — rather than requiring the caller to reconstruct and pass the entire row. Internally, this relies on the newer, more expressive merge mode configuration (rather than a legacy full-record payload class) which understands field-level merge semantics, so it can correctly apply a partial record without accidentally nulling out every column the update didn't mention. This meaningfully reduces the data volume CDC pipelines need to ship and the write amplification Hudi has to absorb for narrow, high-frequency field updates.
More Related questions...