How do you handle errors and failures in a NiFi flow?

NiFi provides several mechanisms for handling failures gracefully, ensuring that failed FlowFiles are not silently lost and that problems are visible to operators.

Failure Relationships: Most processors emit FlowFiles that cannot be processed to a failure relationship. Always connect this relationship to a destination — commonly a LogAttribute processor, a PutFile processor (to archive failed FlowFiles to disk), or a PublishKafka processor (to route failures to an error topic). Never auto-terminate the failure relationship in production without deliberate consideration.

Retry connections: Loop a failure relationship back to the same processor or an earlier processor to implement retry logic. UpdateAttribute can increment a retry counter attribute, and RouteOnAttribute can route FlowFiles with exceeded retry counts to a dead-letter path.

Bulletins: When a processor encounters an error it logs to its bulletin board. Bulletins appear as colored indicators on the processor in the canvas. Severity levels: DEBUG, INFO, WARNING, ERROR.

Yield Duration: If a processor fails to acquire a connection or resource, it enters a yield state for the configured Yield Duration (default 1 second) before being rescheduled again, preventing tight error loops.

Penalty Duration: When a FlowFile is penalized, it is not re-selected for processing for the Penalty Duration period, giving upstream systems time to recover before retry.