If you’re sending the same system prompt on every API call and not using prompt caching, you’re paying full price for tokens the model has already processed. On a production application sending 100,000 requests per month with a 2,000-token system prompt, that’s 200 million tokens you’re overpaying for — potentially hundreds of dollars a month left on the table.
What Prompt Caching Actually Does
Prompt caching lets you pre-process and store a prefix of your prompt on the provider’s infrastructure. When subsequent requests share that same prefix, the provider reuses the cached computation instead of re-processing those tokens from scratch. You pay a fraction of the standard input token rate — and the request completes faster because the model skips the compute-intensive prefill step for the cached portion.
Think of it this way: if your prompt is a 3,000-token system prompt followed by a 200-token user message, and you send 10,000 requests per day, you’re sending 30 million system prompt tokens daily. Without caching, those 30 million tokens are processed fresh every time. With caching, after the first request warms the cache, those 30 million tokens cost roughly 90% less.
The savings compound quickly. Before implementing caching, it’s worth measuring your actual token distribution. The free AI Token Counter shows you exactly how many tokens your system prompt and typical messages use — that breakdown is what determines how much caching will actually save you.
OpenAI Prompt Caching: How It Works
OpenAI introduced automatic prompt caching, meaning you don’t need to explicitly flag what should be cached. The system automatically caches the longest common prefix of your request that meets the minimum token threshold.
Current OpenAI cache pricing (mid-2026):
- GPT-5: $2.50/MTok input → $0.25/MTok cached (90% discount)
- GPT-5.4 Mini: $0.75/MTok input → $0.075/MTok cached (90% discount)
- GPT-4.1: $2.00/MTok input → $0.50/MTok cached (75% discount)
- GPT-4.1 Nano: $0.10/MTok input → $0.025/MTok cached (75% discount)
The cache is stored for approximately 5–10 minutes of inactivity. High-traffic applications with requests coming in constantly will see near-100% cache hit rates. Low-traffic applications or those with long gaps between requests may see partial cache hits.
Minimum prompt length for caching to apply: OpenAI requires the cached prefix to be at least 1,024 tokens. If your system prompt is shorter than that, prompt caching won’t activate. This is worth knowing upfront — a 500-token system prompt gets no cache benefit regardless of request volume.
The implementation from your side is straightforward: there’s nothing to change. If your prompt exceeds 1,024 tokens and you’re sending the same prefix consistently, OpenAI’s API automatically applies cache pricing and returns cache hit indicators in the usage response object (prompt_tokens_details.cached_tokens). Log that field to verify caching is working.
Anthropic Prompt Caching: How It Works
Anthropic takes a different approach — cache control is explicit. You mark specific content blocks for caching using a cache_control parameter in your request. This gives you more control over what gets cached, but requires a small implementation change.
Current Anthropic cache pricing (mid-2026):
- Claude Sonnet 4: $3.00/MTok input → $0.30/MTok cached reads (90% discount), but $3.75/MTok for cache writes (25% premium over standard input)
- Claude Haiku 4.5: $1.00/MTok input → $0.10/MTok cached reads (90% discount), $1.25/MTok cache writes
The cache write premium is the part most teams miss. When a cache entry is created (first request for a given prefix), Anthropic charges 25% more than standard input pricing. Every subsequent request that hits that cache pays only 10% of standard. So the economics depend on how many times you reuse the cache before it expires.
Anthropic’s cache TTL is 5 minutes after the last use. To keep frequently-used caches warm, you may need a lightweight “keepalive” request strategy in low-traffic periods — a design consideration that doesn’t apply with OpenAI’s automatic approach.
Minimum token threshold for Anthropic caching: 1,024 tokens, same as OpenAI. The content block you mark for caching must be at least 1,024 tokens.
Real-World Savings: What the Numbers Look Like
Research published in early 2026 evaluated prompt caching across agentic workflows and found cost savings of 41–80% across providers, with specific results:
- GPT-5.2: 79–81% cost reduction with caching enabled
- Claude Sonnet 4.5: 78–79% reduction
- GPT-4o: 46–48% reduction
- Gemini 2.5 Pro: 28–41% reduction (lower because Gemini’s base pricing is lower, so the absolute savings are smaller)
Time-to-first-token improved 13–31% across providers — a secondary benefit that matters for latency-sensitive applications.
To put this in concrete terms: if you’re spending $1,000/month on a GPT-5-based application, and 70% of your input tokens are in a static system prompt that’s over 1,024 tokens, enabling caching can reduce your monthly bill to roughly $250–300. That’s $700–750 per month saved without changing any business logic or model selection.
The TrueFoundry analysis of provider caching economics makes a useful observation: once caching is enabled, output tokens become the dominant cost line — roughly 58–65% of total cost on typical workloads. This shifts your optimization priorities. After you’ve enabled caching, the next lever is reducing output token volume through tighter instructions and structured output formats.
Which Use Cases Benefit Most from Caching
Caching delivers the biggest savings when three conditions are met: the same prefix is reused frequently, the prefix is long, and the prefix contains static content that doesn’t change between requests.
High-value caching candidates:
Large system prompts with instructions, examples, and rules. A coding assistant might have a 3,000-token system prompt covering code style, available tools, and project context. Cache this and every session starts with nearly zero input cost for that prefix.
Document or knowledge base content. If you’re building a Q&A system over a fixed knowledge base, you can cache the retrieved documents as part of the prompt prefix. A 10,000-token knowledge base prefix cached across 50,000 monthly requests saves roughly 450 million tokens of input compute at standard rates.
Conversation history in long sessions. Anthropic’s explicit cache control lets you cache earlier turns of a conversation so only the most recent turn gets charged at full price. This is especially valuable for coding assistants or research tools where sessions span dozens of turns.
Caching doesn’t help when:
- Your prompt prefix varies significantly between users (personalized system prompts, user-specific context)
- Requests come in too infrequently to keep caches warm
- The cacheable portion is under 1,024 tokens
- You’re doing one-off batch jobs where each prompt is unique
Implementation Gotchas to Avoid
Gotcha 1: Changing the prefix invalidates the cache. Any modification to the cached content — even adding a timestamp, changing a space, or reordering a list — creates a cache miss and triggers a full cache write charge (on Anthropic) or a fresh compute charge (on OpenAI). Keep your static prefix completely static. Move dynamic content (user info, session data) to the end of the prompt, after the cached prefix.
Gotcha 2: Cache warmup cost with Anthropic. The first request for any Anthropic cache entry pays the 25% write premium. For low-frequency requests, the write cost may exceed what you save on cache reads. Do the math: write cost amortized over expected reads should be less than the standard input cost. With a 90% read discount, you break even after roughly 1.3 cache reads per write.
Gotcha 3: Rate limits can bypass caching. If your application exceeds rate limits and requests get queued or retried through different infrastructure, you may see more cache misses than expected. Monitor cache hit rates in your response metadata.
Gotcha 4: Tool and function definitions count toward the cached prefix. This is often overlooked. If you pass a large list of tool definitions on every call, those tokens are included in the cacheable prefix. A set of 15–20 function definitions can easily add 2,000–4,000 tokens to your input. Include them in your static prefix to benefit from caching.
Measure Your Token Costs Before and After
The fastest way to verify that caching is working and actually saving money is to log cached_tokens from your API responses and compare your effective cost-per-request over time. Both OpenAI and Anthropic include cache hit information in the usage field of every response.
Before you implement, get a clear baseline: count your system prompt tokens and estimate your monthly request volume. The free AI Token Counter gives you exact token counts for any prompt — paste your complete system message and representative user input to see the full breakdown. Then run the savings calculation: cached tokens × (standard rate - cached rate) × monthly requests. That number is what’s available to recover with one afternoon of implementation work.
For most production applications sending consistent system prompts, prompt caching is the single highest-ROI optimization available — higher than switching models, higher than prompt compression, higher than architectural changes. It requires no quality tradeoff because the model behavior is identical whether tokens come from cache or fresh compute.
Frequently asked questions
Does prompt caching affect model output quality or behavior? No. The cached tokens produce exactly the same model behavior as fresh processing. The cache stores the internal state (KV cache) after processing those tokens — the model “sees” the same information either way. You will not get different answers because of caching.
How do I know if my prompts are actually being cached?
For OpenAI, check response.usage.prompt_tokens_details.cached_tokens in the API response. A value greater than zero means cache tokens were used. For Anthropic, usage.cache_read_input_tokens tells you how many tokens were served from cache. Log these fields in production and you’ll have real cache hit rate data within hours.
Can I cache different prompts for different users? Yes, but only if the cached prefix is the same across users. The typical pattern is: static system prompt (cacheable) + user-specific context (not cacheable) + user message. Cache the system prompt, send the user-specific content fresh. If your system prompt is fully personalized per user, you lose the caching benefit entirely and should reconsider your prompt architecture.
Does caching work with streaming responses? Yes. Streaming is a response delivery mechanism and doesn’t affect whether input tokens are cached. You can use streaming for real-time UX while still benefiting from cached input tokens.
What’s the breakeven point for Anthropic’s cache write premium? With Anthropic, cache writes cost 25% more than standard input. Cache reads cost 10% of standard input. If standard input is $3.00/MTok, a write costs $3.75/MTok and a read costs $0.30/MTok. You save $2.70/MTok on each cache read versus standard. The write premium is $0.75/MTok above standard. You break even after 0.75 / 2.70 = 0.28 extra reads — meaning you need just one cache read to cover the write cost and come out ahead. In practice, any system with more than 2 requests per cache write benefits from caching.