Integration / Apache NiFi Interview Questions
How does NiFi clustering work and what is the role of ZooKeeper?
A NiFi cluster consists of multiple NiFi nodes that all run the same flow and collectively process data in parallel. Every node receives a copy of the flow definition and runs the same processors, but each node independently processes a subset of the FlowFiles — distributing the workload.
NiFi uses an embedded ZooKeeper (or an external ZooKeeper ensemble) for cluster coordination. ZooKeeper serves two critical roles:
Cluster Coordinator election: One node is elected Cluster Coordinator via ZooKeeper leader election. The Coordinator manages cluster state — which nodes are connected, which are heartbeating, and which are being disconnected. If the Coordinator fails, ZooKeeper elects a new one from the remaining nodes automatically.
Primary Node election: Separately, one node is designated as Primary Node. Processors configured with Execution: Primary Node Only run exclusively on the Primary Node. This is essential for processors that must not run concurrently — ListSFTP or QueryDatabaseTable would produce duplicate FlowFiles if all nodes ran them simultaneously.
All nodes communicate through the Site-to-Site protocol. The NiFi web UI can be accessed on any node and proxies canvas operations to all nodes transparently.
Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!
Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.
Invest now!!! Get Free equity stock (US, UK only)!
Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.
The Robinhood app makes it easy to trade stocks, crypto and more.
Webull! Receive free stock by signing up using the link: Webull signup.
More Related questions...
