Prev Next

BigData / Apache Hudi Interview Questions

What is the difference between snapshot and incremental queries?

A snapshot query answers "what does the table look like right now (or as of a given commit)?" — it reads the full, latest merged state.

Snapshot queryIncremental query
Returns the full current (or point-in-time) table state.Returns only records changed between two specified commits.
Used for typical BI/analytics queries.Used to power incremental ETL and CDC-style downstream pipelines.
Cost scales with table size.Cost scales with the amount of change, not table size.

Incremental queries are what let a downstream job process "just the last hour's changes" cheaply, instead of re-scanning the entire table every run.

A snapshot query's cost scales with:
An incremental query's cost scales with:

More Related questions...

What is Apache Hudi? What does the acronym Hudi stand for? What are the key features of Apache Hudi? What is a data lakehouse, and how does Hudi fit into that model? What are the two table types supported by Hudi? What is Copy-on-Write (CoW) in Hudi? What is Merge-on-Read (MoR) in Hudi? What is a HoodieKey in Hudi? What is the Hudi timeline? What are the main write operations supported by Hudi? What is HoodieStreamer (formerly DeltaStreamer)? What is compaction in Hudi? What is clustering in Hudi? What is the purpose of the cleaner service in Hudi? List the query types supported by Apache Hudi? Define the Hudi Metadata Table? Describe the history and origin of Apache Hudi? What is a payload class in Hudi? Why does Hudi need indexing for upserts? How does the Bloom index work in Hudi? What is the difference between a global index and a partition-level index? How does the Record-Level Index (RLI) improve on the Bloom index? When should you choose Copy-on-Write over Merge-on-Read? When would you choose Merge-on-Read over Copy-on-Write? What is the difference between snapshot and incremental queries? How do you perform a rollback in Hudi? How does a savepoint differ from a rollback in Hudi? Why do MoR tables need periodic compaction? How does Hudi achieve schema evolution? What is the difference between synchronous and asynchronous table services? How can you optimize small file management in Hudi? How do you troubleshoot slow upserts in a Hudi table? What is the difference between optimistic and non-blocking concurrency control in Hudi? When should you use a bucket index instead of a Bloom index? How does the Column Stats index differ from the Bloom filter index? Explain the internal working of an upsert operation in Hudi? Explain the execution flow of a Merge-on-Read compaction? Explain the lifecycle of a Hudi commit on the timeline? How does Hudi's Non-Blocking Concurrency Control work internally? Explain the internal working of the multi-modal indexing subsystem in Hudi? How do you design a partitioning strategy for a very large Hudi table? Explain the internal working of Hudi's incremental query mechanism? How does Hudi ensure ACID guarantees on cloud object storage? Explain the internal working of the LSM-tree-based timeline in Hudi 1.x? How do you set up change data capture ingestion into Hudi? Which is better for high-frequency upserts: Apache Hudi or Apache Iceberg? Explain the internal working of Hudi's file group and file slice model? How do you migrate an existing Parquet-based data lake to Apache Hudi? Explain the internal working of partial updates in Hudi 1.x? How do you tune Hudi for trillion-record-scale upsert workloads?
Show more question and Answers...

Web

Comments & Discussions