BigData / Apache Hudi Interview Questions
What is the purpose of the cleaner service in Hudi?
The cleaner is a background table service that removes older file versions that are no longer needed once they fall outside the table's configured retention policy, reclaiming storage that would otherwise grow unbounded.
Because CoW writes a whole new file version on every update, and MoR periodically compacts into new base file versions, old versions would otherwise accumulate forever. The cleaner respects retention settings so that time-travel queries and incremental consumers that still need recent history aren't broken by files being removed too aggressively.
The cleaner service's main job is to:
Without the cleaner, old file versions would:
More Related questions...
