How does Spring AI support multi-tenancy where different users need different LLM configurations?

Multi-tenancy in Spring AI — where different users, teams, or tenants need different models, API keys, or system prompts — is addressed through a combination of per-request ChatOptions, scoped ChatClient instances, and conversation ID isolation in ChatMemory.

There are three levels at which you can vary configuration per tenant:

1. Different ChatClient instances per tenant — Create a ChatClient per tenant at startup using the same ChatClient.Builder but different defaultSystem prompts and defaultOptions. Store them in a Map<TenantId, ChatClient> and select the right one at request time.

Map<String, ChatClient> clientsByTenant = Map.of(
    "enterprise", builder.defaultSystem("You are an enterprise assistant. Be formal.")
        .defaultOptions(OpenAiChatOptions.builder().withModel("gpt-4o").build()).build(),
    "free", builder.defaultSystem("You are a friendly assistant.")
        .defaultOptions(OpenAiChatOptions.builder().withModel("gpt-4o-mini").build()).build()
);

2. Per-request options override — If tenants only differ in model or temperature, pass .options() per call dynamically based on a resolved tenant context without needing separate client instances.

3. Conversation ID isolation — When using MessageChatMemoryAdvisor, each tenant session uses a unique conversation ID so conversation histories never leak across tenants.

For full API-key-level isolation (e.g. enterprise customers bring their own OpenAI key), you need to construct separate OpenAiChatModel instances with different OpenAiApi clients per key, then wrap each in a ChatClient. Spring AI's auto-configuration does not handle this dynamically at runtime — this requires a custom factory bean.

Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!

Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.

Invest now!!! Get Free equity stock (US, UK only)!

Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.

The Robinhood app makes it easy to trade stocks, crypto and more.

Webull! Receive free stock by signing up using the link: Webull signup.