AI / Core OpenAI Codex Application Fundamentals Interview Questions
What is prompt caching in the OpenAI API and how does it reduce costs?
Prompt caching allows OpenAI's servers to reuse portions of a prompt that were computed in a previous request, reducing both latency and cost. When a significant portion of your prompt matches a cached prefix, you are charged a reduced rate for the cached portion.
How it works: OpenAI automatically caches prompts that share a long common prefix (system prompt, tool definitions, document content). On cache hits, the model skips recomputing those tokens. Cached tokens are cheaper than fresh input tokens.
| Metric | Chat Completions | Responses API |
|---|---|---|
| Cache utilisation improvement | Baseline | 40-80% improvement over Chat Completions (internal tests) |
| Cost reduction | Standard input pricing for all tokens | Discounted rate for cached tokens (e.g. 75% discount for codex-mini-latest) |
| Latency reduction | Baseline | Lower time-to-first-token for repeated prompts |
# The cache works automatically - no special API call needed. # Maximise cache hits by: # 1. Keeping system prompts identical across calls # 2. Placing stable content (instructions, tool definitions) at the START # 3. Placing variable content (user query, session data) at the END # Example: maximise caching for a code review assistant response = client.responses.create( model="gpt-5.5", instructions="""You are an expert Python code reviewer. Follow PEP 8, identify security issues, and suggest improvements. [... 2000 token system prompt stays identical across all calls ...] """, # This large stable prefix gets cached after first call input=f"Review this PR: {variable_code_diff}", # This varies per call ) # codex-mini-latest: 75% prompt caching discount # $1.50/1M -> $0.375/1M on cache hits
Best practices for maximising cache hits: structure prompts with stable content first (system instructions, tool schemas, large documents) and dynamic content last (the user's specific request). The Responses API achieves 40-80% better cache utilisation than Chat Completions due to its state management design.
More Related questions...
