BigData / Apache Airflow Interview Questions
What is the KubernetesExecutor and what are its benefits?
The KubernetesExecutor launches a dedicated Kubernetes pod for every task instance. When a task is scheduled, the executor calls the Kubernetes API to create a pod; the pod runs the task and then terminates. Benefits include:
- Isolation — each task runs in its own container with its own dependencies.
- Resource efficiency — no idle workers; pods start on demand.
- Custom images per task — different Docker images can be specified per task.
- Scalability — Kubernetes handles scheduling and resource allocation.
Configuration is managed through the pod_override parameter or KubernetesPodOperator for full control.
What is the main advantage of KubernetesExecutor over CeleryExecutor for task isolation?
More Related questions...
