BigData / Apache Parquet Interview Questions
What is the Parquet file footer and why does the reader fetch it first?
The Parquet file footer is a serialised Thrift structure at the end of every Parquet file. It contains:
- The full file schema (field names, types, nesting).
- Per-row-group metadata: byte offsets, compressed/uncompressed sizes, row counts.
- Per-column-chunk statistics: min value, max value, null count, distinct count.
- Encoding and compression codec per column chunk.
- Bloom filter offsets (if present).
Readers always fetch the footer first because it is small (typically kilobytes) and provides the complete map needed to plan which row groups and column chunks to read. Without the footer, the reader would have to scan the entire file sequentially.
The last 4 bytes of a Parquet file are the magic bytes PAR1; the 4 bytes before that are a 32-bit integer giving the footer length, so readers seek to the end of the file first.
What are the last 4 bytes of a valid Parquet file?
Why does a Parquet reader seek to the end of the file before reading any data?
More Related questions...
