BigData / Apache Hudi Interview Questions
Explain the internal working of Hudi's file group and file slice model?
A file group is the fundamental storage unit within a partition, identified by a unique fileId. Every record in the table maps to exactly one file group via the index, and that mapping is what makes fast upserts possible — incoming records already know which file group to update.
Within a file group, each commit that touches it produces a new file slice — a base file (Parquet) plus, for MoR tables, zero or more associated log files containing changes not yet compacted into that base file. Over time a file group accumulates multiple slices as new commits and compactions occur; older slices become eligible for removal by the cleaner once they fall outside the retention window. A snapshot query at any point simply reads the latest file slice of every file group as of the query's instant.
More Related questions...
