What is prompt caching?
Quick answer
Prompt caching is an API feature where the provider stores the first part of a prompt it has just processed, such as long instructions or a document, and reuses it on the next request that starts the same way. The repeated part is billed at a lower rate and the answer arrives sooner.
Last updated
Take the free module →Module 5 of the free Voice AI Jump Start course, about 9 minutes. No account needed.
Updated · By Robert Breen
Why it matters for a small business
Many AI setups send the same long text on every call: a system prompt, a price list, a policy document. Without caching you pay full price for that text every time. OpenAI applies caching automatically for long prompts on supported models, and Anthropic lets developers mark which part of a prompt to cache; both bill cached input at a discount.
It is mostly a developer and vendor feature, but it shapes how you design a prompt: put the fixed material first and the changing question last, so the shared start can be reused.
In a real lesson: Give Your Voice Agent Knowledge: A Price List and FAQ in ElevenLabs
In Give Your Voice Agent Knowledge, Riley gets Cedar Creek Plumbing's price list and FAQ. The Retrieval tab shows Prompt only, so both documents go into the prompt in full on every turn. The lesson points out that everything in the prompt adds to the LLM cost of each call, so smaller is better.
That repeated, unchanging text is the situation prompt caching is designed for. The lesson doesn't say whether ElevenLabs caches it, so plan on its advice: keep the documents short, and compare the real cost per call with your estimate in the first month.
Try this lesson free or read the step-by-step guide.
Common confusions
Prompt caching vs memory
Memory saves facts about you for future chats. Caching is a billing and speed shortcut; the model doesn't remember anything extra.
Prompt caching vs RAG
RAG sends only the passages that match each question. Caching sends the whole fixed text but charges less for repeats.
Tips
- Put fixed instructions and documents first, the changing question last.
- Keep the fixed part identical between calls; small edits break the cache.
- Check your provider's usage report for cached tokens.
Related terms
More AI basics terms
Where you use it: free lessons
- Give Your Voice Agent Knowledge: A Price List and FAQ in ElevenLabs (ElevenLabs Agents, 9 min)
- Build a Coverage FAQ Voice Agent for an Insurance Agency (ElevenLabs Knowledge Base) (ElevenLabs Agents, 9 min)
Frequently asked questions
- Does ChatGPT use prompt caching?
- OpenAI offers it in its API. How the ChatGPT app handles it internally isn't something you control or see.
- Do I have to do anything to get prompt caching?
- On some APIs it is automatic for long prompts; on others a developer marks the part to cache. In no-code tools it depends on the vendor.
- Does prompt caching change the answer?
- No. It reuses processing of the same text, so the result is the same as without caching, only cheaper and faster.