What are common LangChain4j anti-patterns to avoid in production applications?

As LangChain4j adoption has grown, several recurring mistakes in production deployments have emerged. Knowing these saves debugging time and prevents costly incidents.

1. Creating ChatLanguageModel or AI Services as request-scoped beans. These are expensive to initialize (TCP connections, key validation, token counting setup). They must be singletons — one instance per application lifecycle, not one per request.

2. Using InMemoryEmbeddingStore in production. Linear scan becomes unacceptably slow above ~50,000 chunks, there is no filtering support, and multiple pods cannot share it. Switch to PgVector or a managed vector DB before going live.

3. Not configuring timeouts. LLM API calls can stall for 60+ seconds. Without a .timeout(Duration.ofSeconds(30)) on the model builder, a hung upstream provider will exhaust your thread pool in a synchronous Spring MVC application.

4. Logging full prompts in production. System messages often contain proprietary business logic. User messages may contain PII. Log only token counts and model names by default; log full prompts only at DEBUG with PII scrubbing.

5. Ignoring ModerationException in safety-critical applications. If you enable @Moderate, surround every AI Services call with a try-catch for ModerationException and return a safe fallback. Uncaught exceptions surface as 500 errors.

6. Embedding the same corpus on every application startup. The ingestion pipeline (load → split → embed → store) should run once and persist results. Re-embedding on startup wastes API budget and delays readiness for large corpora.

7. Hardcoding model names as String literals. Use the constants provided by each provider module (e.g., OpenAiChatModelName.GPT_4_O) so model upgrades are refactor-friendly and typos are caught at compile time.

Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!

Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.

Invest now!!! Get Free equity stock (US, UK only)!

Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.

The Robinhood app makes it easy to trade stocks, crypto and more.

Webull! Receive free stock by signing up using the link: Webull signup.