What is Retrieval-Augmented Generation (RAG) and how does Spring AI implement it?

Retrieval-Augmented Generation (RAG) is the technique of grounding an LLM's answer in documents you provide at query time, rather than relying solely on the model's training data. The model gets injected context that it uses to produce accurate, up-to-date, non-hallucinated responses about your private or recent data — without any fine-tuning.

The workflow has two distinct phases. During ingestion (a one-time or periodic job): load documents → split into chunks → embed each chunk into a vector → store vectors in a vector database. During retrieval (every query): embed the user question → find the top-K most similar chunks in the vector store → inject those chunks into the prompt as context → send to the LLM.

Spring AI components that implement each step:

• DocumentReader — reads source documents (PDF, text, web page, database query).
• TokenTextSplitter — chunks documents to fit embedding and context window limits.
• EmbeddingModel — converts text chunks to float vectors.
• VectorStore — stores and similarity-searches embeddings.
• QuestionAnswerAdvisor — a ChatClient advisor that automates the retrieval + injection step on every call.

// Ingestion (run once)
List<Document> docs = new TokenTextSplitter()
    .apply(new PdfDocumentReader(pdfResource).get());
vectorStore.add(docs);   // embeds internally and stores

// Query-time via advisor (automatic)
ChatClient client = ChatClient.builder(chatModel)
    .defaultAdvisors(new QuestionAnswerAdvisor(vectorStore))
    .build();
String answer = client.prompt().user(question).call().content();