What is document splitting in LangChain4j and why is it necessary?

Document splitting (also called chunking) is the process of dividing a large document into smaller, overlapping segments before embedding and storing them in the vector database. It is a necessary step in RAG pipelines because LLMs have a fixed context window (e.g., 8K, 32K, or 128K tokens). You cannot embed an entire 200-page PDF as a single unit — you need to break it into pieces that fit comfortably in the context window while still carrying enough context to be meaningful.

LangChain4j provides several DocumentSplitter implementations:

• DocumentSplitters.recursive() — Recursively splits on paragraphs, then sentences, then words, aiming to preserve semantic boundaries. This is the recommended default for most text documents.
• DocumentSplitters.byParagraph() — Splits strictly at paragraph boundaries.
DocumentSplitters.bySentence() — Uses sentence boundary detection (requires a sentence detector model).
• DocumentSplitters.byWord(maxTokens) — Splits by word count up to a token limit.

// Recursive splitter: 500 token chunks, 50 token overlap
DocumentSplitter splitter = DocumentSplitters.recursive(500, 50);
List<TextSegment> segments = splitter.split(document);

The overlap parameter is critical: by repeating some tokens at the boundary of adjacent chunks, you ensure that sentences or ideas that span a chunk boundary are not lost in either chunk. Without overlap, a sentence split exactly at a boundary would appear truncated in both chunks, reducing retrieval quality. A 10-20% overlap of the chunk size is a common starting point.