AI / Apache Paimon Interview questions
How does Paimon achieve streaming-batch unification on the same table?
Paimon treats a table's snapshot history as both a bounded dataset and an unbounded stream, depending on how you read it, without needing two separate copies of the data. A batch job simply reads the latest (or a tagged/time-traveled) snapshot as a fixed, bounded result; a streaming job instead subscribes starting from a snapshot and continuously consumes new snapshots as they're committed, effectively treating the table like a queue.
-- Batch read SELECT * FROM my_table; -- Streaming read SELECT * FROM my_table /*+ OPTIONS('scan.mode'='latest') */;
Because both read modes point at the same underlying files and snapshot chain, a Flink streaming job and a Spark batch job can operate on the exact same table concurrently without any synchronization step between "the batch copy" and "the streaming copy" — there's only one copy, and the read mode determines whether it looks like a table or a stream.
More Related questions...
