BigData / Apache Iceberg Interview questions
How do you troubleshoot slow query planning on a large Iceberg table?
Slow query planning on a large Iceberg table generally traces back to a handful of recurring causes, and working through them systematically — usually starting with the metadata tables covered earlier — is faster than guessing blindly at a fix.
- Check for a small-files problem: query the files metadata table to see if the table has accumulated an excessive number of small data files, which inflates the number of manifest entries the planner must evaluate.
- Check manifest count and size: a large number of unconsolidated manifest files (from many small commits over time) similarly increases planning overhead; periodic compaction and manifest rewriting can help.
- Verify partition strategy still fits query patterns: if query filters no longer align well with the table's current partition transform, pruning becomes less effective, and partition evolution may be worth considering.
- Check for excessive delete files under merge-on-read: a high volume of accumulated positional/equality delete files (or lack of deletion vector adoption) can slow both planning and read-time reconciliation.
- Review snapshot retention: an extremely long, unpruned snapshot history can bloat metadata size over time if expiration policies haven't been configured or run.
A useful general discipline is measuring before optimizing: querying the relevant metadata tables (files, manifests, snapshots) to actually confirm which of these factors is the real bottleneck for a specific table, rather than applying maintenance operations speculatively without first understanding which one is actually driving the slow planning time observed.
More Related questions...