What are production deployment patterns for LangChain?

Moving a LangChain application from prototype to production requires addressing reliability, scalability, observability, and cost. The key patterns are:

• LangServe + Docker — wrap chains as FastAPI endpoints with add_routes(), containerise with Docker, deploy to a managed container service (AWS ECS, GCP Cloud Run, Kubernetes). Expose via an API gateway with rate limiting.
• Async endpoints — use ainvoke() / astream() with FastAPI async routes (async def) to handle concurrent requests without blocking worker threads. Pair with uvicorn --workers N or Gunicorn.
• Response caching — use InMemoryCache for same-process caching or SQLiteCache / Redis-backed cache for multi-process. Cache key is the full prompt + model parameters, so identical requests skip the LLM call entirely.
• Observability — enable LangSmith tracing with LANGCHAIN_TRACING_V2=true. Set up alerts on p95 latency and error rate. Track token usage per request to control costs.
• Resilience — apply .with_retry() for transient API errors and .with_fallbacks([cheaper_model]) for budget management under load.
• Secrets management — never hardcode API keys; use environment variables or a secrets manager (AWS Secrets Manager, HashiCorp Vault).