Prompt caching for Claude, GPT and Gemini

When many requests start with the same long text, such as a system prompt, a manual or a code base, the model can reuse what it already read instead of reading it again. Reading from the cache costs less than fresh input and is faster.

Claude models: mark what to cache#

With Claude models, you choose what to cache by adding cache_control to a content block in Messages. Everything up to and including that block becomes the cached prefix:

[
  {
    "type": "text",
    "text": "You are the support assistant for Lumen Notes. Product manual: ...",
    "cache_control": { "type": "ephemeral" }
  }
]

• Put the stable parts first: tools, then instructions, then documents, and the changing question last.
• The cache holds the exact prefix: change one character before the marker and that request reads afresh.
• A short prefix is not cached; caching pays off on long, repeated text.
• cache_control is a Messages feature: in Chat Completions and Responses there is no field to mark the cache for Claude.

A full request with a captured reply is in the caching example.

GPT models: automatic#

GPT models cache long, repeated prefixes on their own; there is nothing to mark. In Responses you can add prompt_cache_key with a stable value per kind of request, so similar requests share the cache.

Gemini models#

There is no cache field to set for Gemini models here. Keep the stable part of your requests at the start, as above.

See it in the usage#

The usage numbers of each reply show what came from the cache:

• Messages: read from the cache in usage.cache_read_input_tokens, written to it in usage.cache_creation_input_tokens.
• Chat Completions: read from the cache in usage.prompt_tokens_details.cached_tokens.
• Responses: read from the cache in usage.input_tokens_details.cached_tokens.

Price#

Cache reads and writes have their own prices per million tokens, shown on each model's page, for example this model's page.