What is prompt caching in the OpenAI API and how does it reduce costs?

Prompt caching allows OpenAI's servers to reuse portions of a prompt that were computed in a previous request, reducing both latency and cost. When a significant portion of your prompt matches a cached prefix, you are charged a reduced rate for the cached portion.

How it works: OpenAI automatically caches prompts that share a long common prefix (system prompt, tool definitions, document content). On cache hits, the model skips recomputing those tokens. Cached tokens are cheaper than fresh input tokens.

# The cache works automatically - no special API call needed.
# Maximise cache hits by:
# 1. Keeping system prompts identical across calls
# 2. Placing stable content (instructions, tool definitions) at the START
# 3. Placing variable content (user query, session data) at the END

# Example: maximise caching for a code review assistant
response = client.responses.create(
    model="gpt-5.5",
    instructions="""You are an expert Python code reviewer.
    Follow PEP 8, identify security issues, and suggest improvements.
    [... 2000 token system prompt stays identical across all calls ...]
    """,  # This large stable prefix gets cached after first call
    input=f"Review this PR: {variable_code_diff}",  # This varies per call
)

# codex-mini-latest: 75% prompt caching discount
# $1.50/1M -> $0.375/1M on cache hits

Best practices for maximising cache hits: structure prompts with stable content first (system instructions, tool schemas, large documents) and dynamic content last (the user's specific request). The Responses API achieves 40-80% better cache utilisation than Chat Completions due to its state management design.

Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!

Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.

Invest now!!! Get Free equity stock (US, UK only)!

Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.

The Robinhood app makes it easy to trade stocks, crypto and more.

Webull! Receive free stock by signing up using the link: Webull signup.