Loading…
Glossary term
Reusing the processed form of a repeated prompt prefix to cut cost and latency.
Many requests start with the same long prefix, like a system prompt or a document. Prompt caching stores the model's processed state for that prefix so later requests skip reprocessing it. Providers often charge much less for cached tokens.
It can cut both cost and delay sharply for apps that reuse large prompts.