Prev Next

BigData / Apache Airflow Interview Questions

What is an Airflow Variable?

Variables are key-value pairs stored in Airflow's metadata database. They provide a way to pass configuration or runtime values into DAGs without hardcoding them. Variables can be set via the UI, CLI, environment variables, or the Python API.

from airflow.models import Variable

# Read a variable (with default fallback)
env = Variable.get('environment', default_var='production')

# Serialize JSON
config = Variable.get('pipeline_config', deserialize_json=True)

For sensitive values, prefer Connections or a secrets backend over plain Variables, since Variable values are stored unencrypted by default.

Are Airflow Variables encrypted by default?

Invest now in Acorns!!! 🚀 Join Acorns and get your $5 bonus!

Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!

Earn passively and while sleeping

Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.

Invest now!!! Get Free equity stock (US, UK only)!

Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.

The Robinhood app makes it easy to trade stocks, crypto and more.


Webull! Receive free stock by signing up using the link: Webull signup.

More Related questions...

What is Apache Airflow? What is a DAG in Apache Airflow? What are Operators in Apache Airflow? What is the Airflow Scheduler? What are the main components of Apache Airflow? What is an Executor in Airflow and what types are available? What is an Airflow Connection? What is an Airflow Variable? What is XCom in Airflow and how is it used? What are Hooks in Apache Airflow? What is the difference between a DAG Run and a Task Instance in Airflow? What are Sensors in Apache Airflow? What is catchup in Airflow and how does it work? What is backfilling in Apache Airflow? What is the TaskFlow API in Airflow? What is the difference between schedule_interval and timetable in Airflow? What is a SubDAG and why is it generally discouraged? What is a TaskGroup in Airflow? What is branching in Airflow and how is BranchPythonOperator used? What are trigger rules in Airflow? What is the Airflow metadata database and what does it store? How does the CeleryExecutor work in Airflow? What is the KubernetesExecutor and what are its benefits? What are Pools in Apache Airflow? What are Airflow Providers? What is dynamic task mapping in Airflow? What is the difference between depends_on_past and wait_for_downstream in Airflow? What is the Airflow Web UI and what can you do with it? What are Airflow task states and what do they mean? What are retries and retry_delay in Airflow tasks? What is a Deferrable Operator (async operator) in Airflow? What are Airflow Plugins? How does Airflow handle templating and macros? What is idempotency in the context of Airflow tasks? What are best practices for writing efficient Airflow DAGs? What is the ExternalTaskSensor in Airflow? What is the KubernetesPodOperator in Airflow? What are SLAs in Apache Airflow and how are they configured? How does Airflow handle task concurrency and parallelism? What is an Airflow Dataset and how does data-driven scheduling work? What is the difference between Airflow and Apache Spark? How do you deploy Apache Airflow using Docker Compose? What is Airflow on Kubernetes (KEDA) autoscaling? What is the SparkSubmitOperator in Airflow? What is Managed Airflow (MWAA) on AWS? How does Airflow handle secrets management? What is the difference between PythonOperator and PythonVirtualenvOperator? What is the Grid view in Airflow 2.x? What are common Airflow anti-patterns to avoid? What is Airflow 2 vs Airflow 1 — key differences?
Show more question and Answers...

Apache Parquet Interview Questions

Comments & Discussions