2.4 Context Window Optimization, Caching & Prompt Reuse

It's tempting to think of a model's context window as having reserved lanes — some space for the system prompt, some for history, some for tool results — the way a suitcase might have a designated pocket for shoes and another for toiletries. That mental model is wrong, and it causes real mistakes. The context window is a single, shared budget, and the system prompt, the conversation history, the tool definitions, the tool results, any retrieved documents, and the model's own response are all drawing from the exact same pool of tokens. Spend more on one and there is, by definition, less available for everything else.

This matters because it reframes the question you should be asking. It's not "do we have room for this tool's full output," as if room were free once the window is large enough. It's "is this tool's full output the best use of the tokens it would occupy, compared to what else is competing for that same space." A system prompt bloated with rarely-needed edge-case instructions, a conversation history nobody has pruned, and a tool that returns forty fields when five matter are all quietly taxing the same shared account.

Here's the counterintuitive part, and it's the single most tested idea in this lesson. You might expect that quality only suffers once you actually run out of room — hit the wall, get truncated, see an error. But quality can degrade well before that, simply because the window is full of low-signal tokens: verbose tool output nobody trimmed, stale history from three topic changes ago, redundant retrieved documents that all say roughly the same thing. This is context rot, sometimes called context drift, and the model doesn't fail loudly when it happens — it just gets worse at finding the thread that actually matters.

Think about trying to find one important sentence in a report that's ballooned to five times its useful length with redundant caveats and repeated boilerplate. You technically have every word available to you, and you could theoretically find the sentence — but in practice, the noise makes you more likely to miss it or misweight it. The model experiences something structurally similar: nothing forces it to ignore the low-signal tokens, so they compete for its attention exactly the way they compete for the token budget.

If the goal is the smallest set of high-signal tokens, you need concrete techniques for getting there, and four of them cover most of the ground — each solving a genuinely different piece of the problem, which is exactly why the exam tests whether you can tell them apart rather than treating them as synonyms.

Pruning drops large or stale tool output that no longer informs the task — the order-lookup result from three steps ago that nobody needs anymore gets removed rather than carried forward indefinitely. Compaction summarizes older turns into a compact recap instead of carrying the full transcript — useful, but remember from the reliability lessons elsewhere in this course that summarization can lose specifics, so what gets compacted matters as much as that it happens. Isolation moves a heavy subtask into a subagent with its own separate context window, so the verbose intermediate work of, say, a research subtask never has to enter the main agent's window at all — only the distilled result comes back. And progressive disclosure loads information as it's actually needed rather than front-loading everything up front, on the theory that information you might need later shouldn't occupy space now.

Isolation is worth pausing on because it connects directly back to the multi-agent orchestration patterns you've studied elsewhere: rather than cramming a heavy exploration task into the primary agent's window and hoping compaction cleans it up later, you delegate it to a subagent up front, and its window absorbs the mess. Progressive disclosure is the same underlying discipline applied earlier in the pipeline — controlling what enters the window in the first place, rather than cleaning up what's already there.

Curation techniques are only useful if you know when to apply them, and that's what the API's usage field is for. Every Messages API response includes a usage field reporting input tokens, output tokens, and cache-related token counts for that call. Tracking it over time gives you the instrumentation to model per-request cost and to catch context bloat creeping up before it actually hits your budget or degrades quality.

It helps to be precise about what usage tracking IS and ISN'T. It is not itself a curation technique — watching a number doesn't prune anything. It's the signal that tells you when pruning, compaction, or isolation is actually needed, the way a fuel gauge doesn't refill your tank but tells you it's time to. A team that tracks input-token trends across a growing conversation type will notice the creeping bloat long before a user notices degraded answers, and can intervene with the right technique from 2.4.3 before context rot sets in.

Now we turn from managing the window's contents to making a well-curated, stable prompt cheap to reuse — Task Statement 2.5's territory, and the direct payoff of the system/user split you learned in Lesson 2.2. When a large stable prefix — a system prompt, a policy document, a few-shot example block — repeats across many requests byte-for-byte, prompt caching lets that prefix be reused instead of reprocessed from scratch every time. Cache reads are cheap; cache writes carry a slight premium. The economics tip in your favor the more requests share that identical prefix — one write, many cheap reads.

The rule that makes this work is the one you already half-learned in Lesson 2.2: order stable content first, dynamic content last. The system prompt and any policy or reference material forms the front of the request; the varying user message comes after it. The longer that uninterrupted stable prefix is, the bigger the cache hit, and a bigger cache hit cuts BOTH time-to-first-token and per-request cost — it's a rare optimization that improves latency and cost at once rather than trading one for the other.

The qualitative framing above — reads are cheap, writes carry a premium — is the durable idea, but an architect sizing an actual cost model needs the approximate magnitudes too, because the size of the effect decides whether caching is worth the added complexity for a given traffic pattern. As illustrative orders of magnitude — verify exact current rates against live Anthropic pricing documentation before finalizing a cost model, since these figures shift as pricing changes — a cache WRITE (the first time a given prefix is cached) costs MORE than an ordinary uncached input token, not less; a cache READ (a later request that hits the same cached prefix) costs roughly a TENTH of the standard input-token rate; and the default cache TTL is approximately FIVE MINUTES, after which an unreused entry expires and the next request pays the write premium again as a fresh write.

The practical consequence is a break-even calculation worth actually running rather than skipping: because the FIRST request on a given prefix pays the write premium (more expensive than uncached) and only LATER requests get the discounted read rate, a prefix reused only once or twice within the TTL window may not save money at all — one expensive write can outweigh a small number of cheap reads. Caching pays off specifically when a stable prefix is reused often enough, and frequently enough within the TTL window, that the accumulated read discount overwhelms the one-time write premium. A high-traffic endpoint reusing the same system prompt thousands of times a minute is an obvious win; a background job that fires the same prompt twice a day, five minutes apart, is not automatically a win — the TTL may have already lapsed between the two calls, forcing a second write instead of a cheap read.

Caching makes ONE prompt cheap to reuse across requests. The last piece of Task Statement 2.5 is about making prompt DESIGN itself reusable across many prompts, many applications, and many teams — because a good system prompt or a good capability shouldn't have to be reinvented every time someone needs something similar.

Modular prompts compose a single prompt from reusable, independently versioned fragments — a role definition, a policy block, a format spec — instead of duplicating the same text across a dozen different prompts. Update the policy fragment once and every prompt that includes it picks up the change, rather than someone hunting down and editing ten near-duplicate blobs of text. This is a maintenance win on its own, and it has a second effect worth noticing: because a change to one fragment doesn't require rewriting the whole prompt, it also helps keep the cacheable prefix stable across revisions.

Agent Skills operate one level up from a single prompt. A Skill packages reusable instructions and procedures — real know-how for accomplishing some capability — so that capability is authored once and then reused across different applications and teams, without standing up a running service to deliver it. Where modular prompts are about composing ONE prompt from fragments, Skills are about packaging a CAPABILITY so many different agents or apps can invoke it without each one re-implementing the same instructions from scratch. It's worth being precise about what a Skill is not: it isn't a deployed microservice or infrastructure component — it's a reuse mechanism for instructions, the prompt-engineering equivalent of a shared library rather than a hosted API.