BigData / Apache Airflow Interview Questions
What is an Executor in Airflow and what types are available?
The Executor defines how tasks are run. Airflow supports several executor types suited for different scale and infrastructure needs:
- SequentialExecutor — runs one task at a time; only for development/testing.
- LocalExecutor — runs tasks in parallel on the same machine using subprocesses.
- CeleryExecutor — distributes tasks across a pool of Celery workers; suitable for production scale-out.
- KubernetesExecutor — spins up a new Kubernetes pod per task; great for isolation and dynamic resource allocation.
- CeleryKubernetesExecutor — hybrid of Celery and Kubernetes executors.
Which executor is recommended only for development and testing?
More Related questions...
