Prev Next

BigData / Apache Parquet Interview Questions

How does AWS Athena query Parquet files in S3?

AWS Athena is a serverless interactive query service that uses Presto/Trino under the hood. It reads Parquet files directly from S3 using the Parquet columnar reader.

Steps to query Parquet in Athena:

  1. Define a Glue Data Catalog table pointing to the S3 prefix:
CREATE EXTERNAL TABLE events (
  user_id BIGINT,
  event_type STRING,
  revenue DOUBLE,
  event_date DATE
)
STORED AS PARQUET
LOCATION 's3://my-bucket/events/'
TBLPROPERTIES ("parquet.compress"="SNAPPY");
  1. Query as normal SQL — Athena applies column pruning and predicate pushdown automatically.

Cost optimisation: Athena charges per byte scanned. Parquet compression + columnar reads can reduce costs by 90%+ compared to querying CSV for the same logical data.

Partition projection or partition-filtered queries (WHERE event_date BETWEEN ...) reduce both scan size and latency.

What query engine powers AWS Athena under the hood?
Why does querying Parquet in Athena cost significantly less than querying CSV for the same data?

Invest now in Acorns!!! 🚀 Join Acorns and get your $5 bonus!

Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!

Earn passively and while sleeping

Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.

Invest now!!! Get Free equity stock (US, UK only)!

Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.

The Robinhood app makes it easy to trade stocks, crypto and more.


Webull! Receive free stock by signing up using the link: Webull signup.

More Related questions...

What is Apache Parquet and why is it used? What are the advantages of Parquet over CSV? How are Parquet files structured? (Row Groups, Column Chunks, Pages)? What is Schema Evolution in Parquet? What is Column Pruning and Projection Pushdown in Parquet? When would you choose Avro over Parquet? How does Parquet handle compression and encoding? What is the Vectorized Reader in Spark and how does it improve Parquet performance? How do you handle schema mismatches when merging multiple Parquet files? If a Spark query on Parquet is slow, what optimisation steps would you take? How do you load Parquet files into Snowflake? What are the supported data types in Parquet? How do you read and write Parquet files in PySpark? How do you read and write Parquet files in Python with PyArrow? What is partitioning in Parquet and how does it improve query performance? What are Bloom Filters in Parquet and when should you use them? What is the difference between Parquet, ORC, and Avro? What is Z-ordering (Z-order clustering) and how does it help Parquet queries? What is Apache Iceberg and how does it use Parquet? How does DuckDB query Parquet files and what makes it fast? What is the Parquet file footer and why does the reader fetch it first? How does Parquet support nested data (structs, lists, maps)? What is small file problem in Parquet-based data lakes and how do you solve it? What is the difference between repartition and coalesce when writing Parquet files? How does AWS Athena query Parquet files in S3? What is predicate pushdown in Parquet and how does it work end-to-end? What are best practices for writing Parquet files in production? How does Google BigQuery use Parquet-style columnar storage internally? What is Delta Lake and how does it extend Parquet for ACID transactions? How do you perform upserts (MERGE INTO) on Parquet-based tables in Delta Lake?
Show more question and Answers...

Web

Comments & Discussions