How does Spring AI's retry and resilience mechanism work for LLM API calls?

Network-level failures and provider rate limits are unavoidable when calling external AI APIs. Spring AI integrates with Spring Retry to automatically retry failed model calls using exponential backoff, shielding application code from transient errors.

Retry is enabled per provider via properties. For OpenAI:

spring.ai.retry.max-attempts=3
spring.ai.retry.backoff.initial-interval=2000    # ms
spring.ai.retry.backoff.multiplier=2.0
spring.ai.retry.backoff.max-interval=30000       # ms
spring.ai.retry.on-client-errors=false           # do NOT retry 4xx errors (bad prompt, wrong model)
spring.ai.retry.exclude-on-http-codes=401,403    # skip retrying auth errors

The default behaviour retries on HTTP 429 (rate limit), 503 (service unavailable), and network-level IOExceptions. It intentionally does not retry 4xx client errors like 400 (bad request) or 401 (unauthorised) because retrying these would waste quota and always fail again. The on-client-errors=false property enforces this.

For structured output calls, a parse failure (the LLM returns malformed JSON) is not an HTTP error so Spring Retry won't catch it automatically. The recommended pattern is to wrap .entity() calls in a RetryTemplate or use a @Retryable annotated service method that retries with a more explicit prompt on JsonProcessingException.

Beyond retry, circuit breaker integration (Resilience4j) is a natural complement for protecting downstream services when a provider is consistently failing. This is not built into Spring AI itself but layers on top of the standard Spring Cloud Circuit Breaker abstraction.