BigData / Apache Airflow Interview Questions
What is the difference between depends_on_past and wait_for_downstream in Airflow?
Both parameters control inter-run dependencies for a task:
- depends_on_past=True: The task will only run if the same task in the previous DAG Run succeeded. Useful for tasks that process sequential data.
- wait_for_downstream=True: The task will wait until all immediately downstream tasks from the previous DAG Run have completed. Stricter than
depends_on_past.
These are set as operator-level parameters and are applied per task, not per DAG.
More Related questions...