AI / Apache Paimon Interview questions
What happens when Paimon's sorted runs contain overlapping primary key ranges?
Overlap between sorted runs is expected and normal — it's how new writes get in without rewriting old files. The cost shows up at read time: a query has to open every sorted run whose key range could contain the requested key(s), then merge whatever records it finds for each key using the table's merge engine, honoring sequence.field ordering if configured.
The more sorted runs pile up with overlapping ranges, the more files a read has to touch and merge, which slows queries down. That's precisely the problem compaction exists to solve: it periodically merges several sorted runs together into fewer, non-overlapping ones, reducing the read-amplification cost back down. Left unchecked, an LSM tree with too many un-compacted sorted runs is the classic cause of degraded read performance on a heavily-streamed table.
More Related questions...
