What are chain fallbacks and retries?

Fallbacks and retries are resilience mechanisms built into LangChain Runnables that make production chains tolerant of transient failures and model quality issues.

Fallbacks — .with_fallbacks() attaches one or more backup Runnables that are tried in order if the primary raises an exception. You can fall back to a cheaper model, a different provider, or a static response:

from langchain_openai import ChatOpenAI
from langchain_anthropic import ChatAnthropic

chain = (
    ChatOpenAI(model="gpt-4o")
    .with_fallbacks([
        ChatAnthropic(model="claude-3-sonnet-20240229"),
        ChatOpenAI(model="gpt-4o-mini"),
    ])
)
# If gpt-4o fails, tries Claude; if that fails, tries gpt-4o-mini

Retries — .with_retry() retries the same Runnable on failure with configurable stop conditions and wait strategies:

from langchain_openai import ChatOpenAI

resilient_llm = ChatOpenAI().with_retry(
    retry_if_exception_type=(RateLimitError, Timeout),
    stop_after_attempt=3,
    wait_exponential_jitter=True,
)
chain = prompt | resilient_llm | StrOutputParser()

You can combine both: retry first (for transient errors), then fall back (if the model is genuinely unavailable). Retries are best for rate-limit errors; fallbacks are best for model outages or quality failures (e.g. the primary model produces invalid JSON).

Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!

Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.

Invest now!!! Get Free equity stock (US, UK only)!

Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.

The Robinhood app makes it easy to trade stocks, crypto and more.

Webull! Receive free stock by signing up using the link: Webull signup.