Prompt Caching & Cache Checkpointing

How Prompt Caching Works

When many requests share a large, stable prefix -- a system prompt, a long reference document, tool definitions -- that prefix can be cached instead of being reprocessed at full price on every call. You mark the end of the stable prefix with a cache_control field of type "ephemeral" on the last content block you want cached -- that field marks a breakpoint, and everything up to and including that block gets cached as a unit. You can place up to 4 breakpoints per request, which matters for prompts with multiple stable segments (e.g., a stable tool-definition block plus a separately stable, longer-lived reference document appended after it).

Breakpoint placement has to respect one structural fact: the API always processes a request in a fixed order -- tools first, then the system prompt, then messages. A breakpoint placed after the tool definitions caches the tools alone; a breakpoint placed at the end of the system prompt caches tools *and* system together (since both preceded it); a breakpoint inside messages caches everything before it in that fixed order. You cannot cache messages content "before" the system prompt, because messages are never processed before the system prompt regardless of where you set a breakpoint.

The Exact Rates -- Not Just "Reads Are Cheap, Writes Cost More"

The two rates in that structure point in opposite directions, and the exam expects you to know the actual multipliers, not just their direction:

• Cache writes are billed at a premium over the base input-token price: 1.25x for the default 5-minute TTL, or 2x for the opt-in 1-hour TTL.
• Cache reads cost only 0.1x the standard input-token price -- a 90% discount.

The economics only pay off through reuse within the TTL window: the call that writes the cache pays a premium (1.25x or 2x), and every subsequent call that reads that same cached prefix before it expires pays only 0.1x. A single one-off call that writes a cache and is never read again pays the write premium with nothing to offset it -- that call costs strictly MORE than the same call would have cost with no caching at all. This is a favorite exam trap: caching a prompt you only ever send once is a net loss, not a neutral no-op.

Worked Example: When Does Caching Actually Pay Off?

Suppose a 2,000-token system prompt is stable and reused across 50 calls, all made within a single 5-minute window (each read resets the cache's 5-minute clock, so the cache stays alive throughout a burst like this).

Caching this reused prefix costs roughly 12% of the uncached cost across the 50-call burst -- an ~8x reduction, driven entirely by the 49 cheap reads offsetting the single write premium. Contrast this with a single one-off call: 1 write at 1.25x (2,500 units) costs 25% MORE than the uncached 1x cost (2,000 units) would have -- exactly the exam-trap scenario above, made concrete.

Cache Lifetime: 5-Minute Default, 1-Hour Opt-In

The default cache lifetime is 5 minutes from the last read -- every read resets that clock, so a prefix that keeps getting hit stays alive indefinitely without ever needing to be rewritten. Let 5 minutes pass with no read and the cache expires; the next call pays the write premium again. For workloads with gaps between calls longer than 5 minutes (an agent that pauses between steps, a workflow with irregular traffic), you can opt into a 1-hour TTL by setting ttl: "1h" on the breakpoint -- at the higher 2x write cost noted above. Choosing 1-hour TTL is a bet that fewer, more-expensive writes plus surviving longer gaps beats more-frequent 5-minute writes triggered by expiration.

There's a Minimum Token Threshold

Caching only applies above a minimum token threshold -- roughly 1,024 tokens for most current models (this varies by model). A breakpoint set on a prompt shorter than that threshold simply will not cache: no error is raised, the request just proceeds as if cache_control were never set. This means a cache_control field on a short prompt is not a correctness issue, but it is a silent no-op -- don't assume caching is happening just because the field is present in the request.

Cache Checkpointing

For a prompt that grows incrementally over a session (e.g., an agent loop that keeps appending tool results to a running context), cache checkpointing lets you cache at multiple points along that growing prompt (using some of those same 4 available breakpoints), so a long, incrementally-extended context stays mostly cached even as new content is appended at the end -- rather than invalidating the whole cache on every extension.

Don't Truncate a Stable Prefix Instead of Caching It

A common mistake is truncating a needed document to save tokens when caching it would have preserved the full content at a similar or better cost profile -- truncation trades away correctness for a saving that caching could deliver without any information loss.

Common exam traps

• "Cache reads and writes are priced the same" -- they are not; reads are 0.1x (a 90% discount) and writes are 1.25x (5-min TTL) or 2x (1-hour TTL), and caching only pays off through reuse across multiple calls, not on the first call.
• "A single cached call is never worse than an uncached one" -- false. A one-off call that writes a cache and is never read again costs strictly MORE (1.25x or 2x) than the same call with no caching at all (1.0x), because it pays the write premium with no read to offset it.
• Forgetting the fixed processing order (tools, then system, then messages) when reasoning about what a given breakpoint actually caches.
• Assuming a cache_control field always caches something -- below the ~1,024-token minimum (model-dependent), it silently does nothing.
• Assuming a single large document should just be truncated to control cost, missing that caching the full document is usually the better move when it will be reused.