How do you stream responses from an LLM in Spring AI?

Streaming in Spring AI lets you consume LLM output token-by-token as a reactive Flux<String> (or Flux<ChatResponse> for full metadata) rather than waiting for the entire response to arrive. This is critical for chat UIs where users expect to see text appear progressively.

Replace .call() with .stream() in the ChatClient chain:

// Stream plain text tokens
Flux<String> tokenStream = chatClient.prompt()
    .user("Write a short story about a Java developer.")
    .stream()
    .content();

// Consume in a WebFlux controller
@GetMapping(value = "/story", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<String> story() {
    return chatClient.prompt()
        .user("Tell me a story")
        .stream()
        .content();
}

For full response metadata (finish reason, token usage per chunk) use .stream().chatResponse() which returns Flux<ChatResponse>. If you need to collect the complete text after streaming for post-processing, use the standard Project Reactor .collectList() or .reduce() operators.

When using the lower-level ChatModel interface directly, call chatModel.stream(prompt) which also returns Flux<ChatResponse>. Note that not every provider supports streaming — check the provider's documentation. OpenAI, Anthropic, and Ollama all support it; some Bedrock models do not.

Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!

Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.

Invest now!!! Get Free equity stock (US, UK only)!

Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.

The Robinhood app makes it easy to trade stocks, crypto and more.

Webull! Receive free stock by signing up using the link: Webull signup.