Explain how Iceberg integrates with Apache Spark for reading and writing?

Spark integrates with Iceberg through a dedicated catalog plugin and SQL extensions, letting standard Spark SQL and DataFrame operations transparently read from and write to Iceberg tables once the appropriate catalog is configured, without needing Iceberg-specific application code for basic operations.

spark = SparkSession.builder \
    .config("spark.sql.catalog.my_catalog", "org.apache.iceberg.spark.SparkCatalog") \
    .config("spark.sql.catalog.my_catalog.type", "rest") \
    .config("spark.sql.catalog.my_catalog.uri", "https://catalog.example.com") \
    .getOrCreate()

spark.sql("SELECT * FROM my_catalog.db.events WHERE event_time >= '2026-01-01'")
df.writeTo("my_catalog.db.events").append()

Once a Spark session is configured with an Iceberg catalog (Hive, Glue, REST, or others), standard Spark SQL statements against tables under that catalog automatically go through Iceberg's query planning and commit logic — Spark's own SQL and DataFrame APIs remain the interface, while Iceberg's Spark integration handles translating those operations into the correct manifest reading, pruning, and snapshot commit behavior underneath.

Beyond basic reads and writes, Spark's Iceberg integration also exposes Iceberg-specific SQL extensions for operations that don't have a standard SQL equivalent — time travel syntax, calling maintenance procedures like rewrite_data_files or expire_snapshots, and branch/tag management — all accessible through ordinary spark.sql() calls once the relevant Iceberg Spark extensions are enabled in the session configuration.