Prompt caching
Definition
Prompt caching is an AI API feature that stores the processed start of a prompt you send repeatedly, so later requests beginning with exactly the same text are cheaper and faster. Only the repeated beginning is reused; the answer is still written fresh each time.
By Ronan Liedmeier, fractional head of growth and course author · Published 17 October 2026 · Last updated 17 October 2026
Also called: context caching
It pays off in any workflow that sends the same long instructions or reference document on every call.
How it works
Once caching is on (automatic at OpenAI, a request setting at Anthropic), the provider checks whether the opening of each prompt matches something it processed recently. If it does, it reuses that work and bills those input tokens at a reduced rate. If not, it processes the full prompt and keeps the opening for next time. Anthropic and OpenAI both document that the match has to be exact: change one character near the top and everything after it is processed again. Cached entries also expire after a period without use.
That makes order important. Put stable content first (the system prompt, examples, reference material) and content that changes last (the user's question, today's date, the record being processed).
Example
A newsletter workflow sends the same long style guide plus one new article to a model dozens of times a day. The original prompt started with the article and ended with the style guide, so nothing was ever reused. Moving the style guide to the top and the article to the end lets the provider reuse the style guide on every call while the cache is warm, and the input bill drops.
Common mistake
Putting something that changes on every call at the top of the prompt, such as a timestamp, a customer name or a record ID. That breaks the exact match, nothing is reused, and you pay full price on every request without noticing. When API costs look high, check where your variable fields sit before you switch to a cheaper model.
Learn it in AI Academy
- Cost & Latency AwarenessAdvanced · 13 min
- Advanced Prompting in 2026Expert · 7 min
Related terms
- Token
- A token is the small chunk of text an AI language model reads and writes, usually a short word or a piece of a longer one. In English one token averages about three-quarters of a word, and context windows and API prices are counted in tokens.
- Latency (AI response time)
- Latency is the time an AI system takes to respond, from the moment a request is sent until the answer, or its first words, arrive. In agent workflows it adds up across every model call and tool call in a run.
- System prompt
- A system prompt is a set of standing instructions given to an AI model before the conversation starts, telling it how to behave in every reply. It sets the model's role, tone, rules and output format, and people chatting with the assistant usually never see it.
- API (application programming interface)
- An API (application programming interface) is a set of requests a software service agrees to accept from other programs, a bit like a menu. A program sends a request such as "create a contact" and the service sends back data or a confirmation.