How does Spring AI's retry and resilience mechanism work for LLM API calls?

Network-level failures and provider rate limits are unavoidable when calling external AI APIs. Spring AI integrates with Spring Retry to automatically retry failed model calls using exponential backoff, shielding application code from transient errors.

Retry is enabled per provider via properties. For OpenAI:

spring.ai.retry.max-attempts=3
spring.ai.retry.backoff.initial-interval=2000    # ms
spring.ai.retry.backoff.multiplier=2.0
spring.ai.retry.backoff.max-interval=30000       # ms
spring.ai.retry.on-client-errors=false           # do NOT retry 4xx errors (bad prompt, wrong model)
spring.ai.retry.exclude-on-http-codes=401,403    # skip retrying auth errors

The default behaviour retries on HTTP 429 (rate limit), 503 (service unavailable), and network-level IOExceptions. It intentionally does not retry 4xx client errors like 400 (bad request) or 401 (unauthorised) because retrying these would waste quota and always fail again. The on-client-errors=false property enforces this.

For structured output calls, a parse failure (the LLM returns malformed JSON) is not an HTTP error so Spring Retry won't catch it automatically. The recommended pattern is to wrap .entity() calls in a RetryTemplate or use a @Retryable annotated service method that retries with a more explicit prompt on JsonProcessingException.

Beyond retry, circuit breaker integration (Resilience4j) is a natural complement for protecting downstream services when a provider is consistently failing. This is not built into Spring AI itself but layers on top of the standard Spring Cloud Circuit Breaker abstraction.

Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!

Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.

Invest now!!! Get Free equity stock (US, UK only)!

Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.

The Robinhood app makes it easy to trade stocks, crypto and more.

Webull! Receive free stock by signing up using the link: Webull signup.