How do you handle LLM output parsing failures gracefully in LangChain4j?

When LangChain4j requests structured output (returning a POJO from an AI Services method), the LLM occasionally produces malformed JSON despite format instructions — especially with smaller models or complex schemas. Without explicit error handling, this surfaces as a OutputParsingException or JsonParseException from Jackson. Graceful handling is critical for production reliability.

There are three layers where you can handle parsing failures:

1. Return Optional to signal missing/failed results:

interface ReviewExtractor {
    Optional<ProductReview> extractReview(String rawText);
}
// Returns Optional.empty() if parsing fails (safer than exception-based control flow)

2. Catch OutputParsingException at the call site and fall back:

try {
    ProductReview review = extractor.extractReview(text);
    return review;
} catch (OutputParsingException e) {
    log.warn("Failed to parse review structure: {}. Falling back to raw text.", e.getMessage());
    return ProductReview.unparsed(text); // your fallback model
}

3. Retry with an explicit correction prompt:

@Retryable(retryFor = OutputParsingException.class, maxAttempts = 2)
ProductReview extractWithRetry(String text) {
    return extractor.extractReview(text);
}

Reducing parsing failures proactively:

• Use providers with native JSON mode (OpenAI's response_format: json_object) — configure via OpenAiChatModelName and set responseFormat on the model builder
• Add few-shot examples of correct JSON structure in the system message
• Use simpler schemas — fewer fields, no deeply nested objects, enums instead of free-text strings for constrained values
• Use a more capable model for extraction tasks where schema adherence is critical