What are Sensors in Apache Airflow?

Sensors are a special type of operator that wait for a certain condition to become true before proceeding. They poke the condition at a configurable interval (poke_interval) and either block (poke mode) or release the worker slot between polls (reschedule mode).

Common sensors:

• FileSensor — waits until a file appears on the filesystem.
• S3KeySensor — waits for an object key to exist in S3.
• HttpSensor — polls an HTTP endpoint until a condition is met.
• ExternalTaskSensor — waits for a task in another DAG to succeed.

Using mode='reschedule' is best practice for long waits to avoid holding worker slots idle.

Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!

Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.

Invest now!!! Get Free equity stock (US, UK only)!

Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.

The Robinhood app makes it easy to trade stocks, crypto and more.

Webull! Receive free stock by signing up using the link: Webull signup.