BigData / Apache Airflow Interview Questions
What is the TaskFlow API in Airflow?
Introduced in Airflow 2.0, the TaskFlow API is a decorator-based approach to writing DAGs and tasks that reduces boilerplate. Instead of instantiating operator objects explicitly, you decorate plain Python functions with @task and define the DAG with @dag.
from airflow.decorators import dag, task from datetime import datetime @dag(schedule='@daily', start_date=datetime(2024, 1, 1), catchup=False) def my_pipeline(): @task def extract(): return {'rows': 100} @task def transform(data: dict): return data['rows'] * 2 transform(extract()) my_pipeline()
XCom passing is handled automatically when the return value of one @task is passed as an argument to another.
Which Airflow version introduced the TaskFlow API?
More Related questions...
