A content agency running GPT-4o at $3,200/month cut their bill to $480 in six weeks without changing models or reducing output volume. Every tactic they used is in this guide — and most require less than a day of implementation.
Start Here: Measure Before You Optimize
The single biggest mistake teams make is implementing cost optimizations before they know where the tokens are going. You can’t prioritize what you haven’t measured.
Before applying any of the tactics below, pull 30 days of data from your OpenAI usage dashboard (platform.openai.com/usage) and categorize your calls by workflow type. In almost every case, 20% of your call types are consuming 70–80% of your token costs. Those are the only ones worth optimizing first.
For each high-cost workflow, paste a representative prompt-plus-response pair into the free AI Token Counter to get the exact token count. Multiply by daily call volume to see your monthly token footprint per workflow. This takes about an hour and turns guesses into numbers you can actually optimize against.
Tactics 1–5: Reduce Input Tokens
1. Shorten your system prompt. This is consistently the highest-leverage change. Most system prompts contain redundant instructions, example scenarios that could be removed, and verbose phrasing that conveys nothing extra. A 2,000-token system prompt rewritten to 400 tokens — with identical behavior — saves 1,600 tokens per API call. At 10,000 calls/day on GPT-4o, that’s 16 billion tokens/month, or roughly $80,000 in annual savings on input costs alone.
How to audit your system prompt: paste it into the AI Token Counter, then strip any sentence that doesn’t change the model’s behavior. Test empirically — remove a clause, run 20 test prompts, check if output quality degrades.
2. Prune conversation history aggressively. Many chat applications pass the full conversation history with every message. A 10-turn conversation with 500 tokens per turn sends 5,000 extra tokens per message by turn 10. Strategies: keep only the last N turns (3–5 is usually sufficient), use a running summary that compresses older context, or inject only the most relevant prior turns rather than all of them.
3. Remove whitespace and formatting from API inputs. JSON with pretty-printing uses 20–30% more tokens than compact JSON. If you’re passing structured data to the API, serialize it without indentation. Same principle for any structured input format.
4. Trim retrieved context in RAG pipelines. Retrieval-augmented generation pipelines often over-retrieve context to be safe, then pass too much of it to the model. If you’re retrieving 10 chunks of 500 tokens each and the model only needs 2–3 to answer correctly, you’re wasting 3,500–4,000 input tokens per call. Reduce chunk count, add a relevance threshold before inclusion, or use a fast cheap model to pre-filter retrieved context.
5. Compress examples in few-shot prompts. Few-shot examples are expensive because they’re repeated on every call. Two well-chosen examples almost always outperform five mediocre ones. If your prompt has 5+ examples, remove them one at a time and test — you’ll often find 2–3 are carrying all the weight.
Tactics 6–10: Reduce Output Tokens
6. Specify output length explicitly. The single most reliable way to reduce output token costs is to instruct the model with exact length constraints: “Respond in 3 sentences or fewer.” “Your output should be a JSON object with exactly these fields.” “Write a 150-word summary.” Without length constraints, models default to over-generating.
7. Use structured output formats. JSON output is more token-efficient than prose for structured data. A JSON object with 5 fields typically uses fewer tokens than an equivalent paragraph describing those 5 fields, and it eliminates the need for downstream parsing.
8. Eliminate model preamble in the output. By default, models often begin responses with “Certainly, here’s the answer…” or “Great question.” These conversational openers consume tokens and carry no information. Add to your system prompt: “Begin responses directly without introductory phrases or acknowledgments.”
9. Request concise reasoning when using chain-of-thought. If you need the model to reason through a problem, instruct it to reason concisely. “Think step by step, but keep your reasoning to 3–5 bullet points before answering” often produces equivalent accuracy to unconstrained chain-of-thought at a fraction of the token cost.
10. Use streaming and stop sequences. If your application processes the response as it streams in, you can detect when the model has included all required information and stop the generation early. Stop sequences let you define a string that terminates the response — useful for structured workflows where the output has a clear completion marker.
Tactics 11–15: Model Routing and Caching
11. Route tasks to the cheapest capable model. GPT-4o mini costs roughly 30× less than GPT-4o. For many well-defined tasks — classification, simple extraction, FAQ response, short-form content — mini is indistinguishable from GPT-4o on output quality. Implement a routing layer that sends simple, well-structured tasks to mini and escalates complex ones to GPT-4o or GPT-4o Plus. This routing pattern, applied correctly, typically reduces costs by 40–60% without degrading user-facing quality.
12. Use GPT-4o mini for first-pass filtering. If you have a pipeline that processes all inputs through an expensive model, add a cheap filtering step first. GPT-4o mini can determine in 100–200 tokens whether a request needs GPT-4o’s capabilities. The filter step costs a fraction of a cent; routing the wrong inputs to the expensive model costs much more.
13. Implement prompt caching. OpenAI’s prompt caching (available for GPT-4o and o-series models) automatically caches the prefix of your prompt when it meets length requirements and gets reused frequently enough. Cached tokens cost 50% less than uncached tokens. To maximize cache hit rate: keep your system prompt at the beginning of every request, make it static (don’t embed dynamic variables in the system prompt), and ensure your context length exceeds the caching threshold (currently 1,024 tokens minimum).
14. Cache responses for repeated queries. If your application serves similar queries to multiple users, a semantic cache layer (using a vector store to match new queries to prior responses) can dramatically reduce API calls. A customer support bot where 40% of questions are variations of the same 20 questions should see 40% call reduction from caching. Libraries like GPTCache or a Redis-based semantic similarity layer implement this without much overhead.
15. Use batch processing for non-real-time workloads. OpenAI’s Batch API processes requests asynchronously with a 24-hour turnaround and charges 50% less than the synchronous API. Any offline workload — nightly data enrichment, document processing queues, scheduled content generation — should default to the Batch API. The 50% discount applies to all models, including GPT-4o.
The Compounding Effect: Stack the Tactics
These tactics multiply, not just add. A workflow where you trim the system prompt (saves 60% of input tokens), route 70% of calls to GPT-4o mini, and enable batch processing on the remaining GPT-4o calls can produce total cost reductions of 85–92% — even when individual tactics each contributed 30–50% in isolation.
The agency example from the opening: they trimmed system prompts (cut input tokens by 65%), routed classification tasks to mini (reduced GPT-4o call volume by 70%), and enabled batch processing for their overnight content generation runs (50% off remaining calls). Three tactics, six weeks, $2,720/month saved.
See Your Token Count Before Optimizing
You can’t accurately estimate cost savings without knowing your current token consumption. Paste your existing system prompt, a typical user message, and a representative model response into the free AI Token Counter — it returns the exact token count plus a monthly cost projection at your call volume. Run this before and after applying each tactic to measure actual savings, not estimated savings.
Frequently Asked Questions
How much of a cost reduction is realistic for most teams? Based on patterns across NMM students who have run optimization projects, teams with unoptimized workflows — meaning system prompts haven’t been audited, all calls go to the same model, and there’s no batch processing — typically achieve 50–75% cost reduction within the first two weeks. The 90%+ reductions happen when model routing and caching are layered on top.
Does shortening prompts reduce output quality? It depends on what you cut. Removing genuinely redundant instructions, verbose phrasing, and rarely-exercised examples rarely degrades quality. Removing constraint instructions, output format specifications, or context that the model actually uses will degrade quality. The only reliable answer is empirical testing on your actual workloads.
What’s the minimum prompt length for OpenAI’s prompt caching to activate? Currently 1,024 tokens. Your system prompt and any static prefix content need to exceed this threshold for caching to engage. This is worth knowing because some teams have short, efficient system prompts that don’t qualify — in that case, other tactics apply instead.
Can I use all these tactics with GPT-4o mini, not just GPT-4o? Yes. All 15 tactics apply to any OpenAI model. The percentage savings differ by model (prompt caching is more valuable on expensive models), but the principles hold across the model lineup.
Is there a risk of the model ignoring instructions if the system prompt is too short? Not inherently — model performance depends on instruction quality, not length. A 200-token system prompt with clear, specific instructions often outperforms a 2,000-token system prompt with repetitive or contradictory instructions. Specificity and testability matter more than length.