How does Spring AI support multi-tenancy where different users need different LLM configurations?

Multi-tenancy in Spring AI — where different users, teams, or tenants need different models, API keys, or system prompts — is addressed through a combination of per-request ChatOptions, scoped ChatClient instances, and conversation ID isolation in ChatMemory.

There are three levels at which you can vary configuration per tenant:

1. Different ChatClient instances per tenant — Create a ChatClient per tenant at startup using the same ChatClient.Builder but different defaultSystem prompts and defaultOptions. Store them in a Map<TenantId, ChatClient> and select the right one at request time.

Map<String, ChatClient> clientsByTenant = Map.of(
    "enterprise", builder.defaultSystem("You are an enterprise assistant. Be formal.")
        .defaultOptions(OpenAiChatOptions.builder().withModel("gpt-4o").build()).build(),
    "free", builder.defaultSystem("You are a friendly assistant.")
        .defaultOptions(OpenAiChatOptions.builder().withModel("gpt-4o-mini").build()).build()
);

2. Per-request options override — If tenants only differ in model or temperature, pass .options() per call dynamically based on a resolved tenant context without needing separate client instances.

3. Conversation ID isolation — When using MessageChatMemoryAdvisor, each tenant session uses a unique conversation ID so conversation histories never leak across tenants.

For full API-key-level isolation (e.g. enterprise customers bring their own OpenAI key), you need to construct separate OpenAiChatModel instances with different OpenAiApi clients per key, then wrap each in a ChatClient. Spring AI's auto-configuration does not handle this dynamically at runtime — this requires a custom factory bean.