What are the Spring AI Chat Model options for controlling response determinism?

Response determinism in LLMs is primarily controlled through two inference parameters: temperature and top-p (nucleus sampling). Both are set via ChatOptions in Spring AI and work together to shape how randomly the model selects the next token at each step of generation.

Temperature scales the probability distribution over the vocabulary before sampling. A temperature of 0.0 makes the model almost always choose the single highest-probability token (near-deterministic, repetitive). A temperature of 1.0 samples from the raw distribution. Values above 1.0 flatten the distribution further, increasing creativity and randomness. For factual tasks (Q&A, code generation, data extraction) use 0.0–0.3. For creative tasks (writing, brainstorming) use 0.7–1.0.

Top-p restricts sampling to the smallest set of tokens whose cumulative probability exceeds p. A top-p of 0.9 means the model only considers tokens that together account for 90% of the probability mass, discarding long-tail unlikely tokens. Most practitioners either tune temperature alone and leave top-p at 1.0, or tune top-p alone and leave temperature at 1.0 — adjusting both simultaneously is rarely necessary and harder to reason about.

// Deterministic (code analysis, data extraction)
ChatOptions factual = ChatOptionsBuilder.builder()
    .withTemperature(0.1f).withTopP(1.0f).build();

// Creative (story, marketing copy)
ChatOptions creative = ChatOptionsBuilder.builder()
    .withTemperature(0.85f).withTopP(0.95f).build();

Note that even temperature 0 is not fully deterministic across all providers due to floating-point parallelism in GPU computations — you may see occasional token variation on identical inputs.

Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!

Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.

Invest now!!! Get Free equity stock (US, UK only)!

Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.

The Robinhood app makes it easy to trade stocks, crypto and more.

Webull! Receive free stock by signing up using the link: Webull signup.