Context Compression
Context ManagementDefinition
The practice of actively reducing a conversation's token footprint so it fits within the model's context window without silent truncation. Encompasses multiple strategies — rolling window eviction, progressive summarization, external storage with retrieval, and prompt caching — each with different loss profiles and complexity trade-offs. Understanding this menu of options, and knowing what must never be compressed, is a core Domain 5 skill.
Example Usage
An agentic loop monitors token usage; at 75% of the context limit it triggers phase-level summarization of completed tasks, retains verbatim the last 10 turns, and never compresses active tool results or live constraints.
Why It Matters for the CCA-F Exam
The CCA-F exam tests context compression as a Domain 5 (Context Management & Reliability) strategy. Expect scenario questions where an agentic loop is approaching the context limit — you must identify the correct compression approach and flag what should *not* be compressed (active tool results, live constraints, partially completed tasks). The comparison table below is exam-relevant.
In Depth
Every interaction with Claude occupies tokens: the system prompt, prior messages, tool results, and the pending user turn all compete for finite space. Once the context window fills, something must give — either old turns are truncated silently, the API returns model_context_window_exceeded, or you control the reduction yourself. Context compression puts that control in your hands.
For long-running agentic loops, compression is not optional — it is infrastructure.
The menu of strategies
1. Rolling window eviction — retains only the N most recent turns, dropping older ones as new ones arrive. Lowest complexity, highest information loss. Safe for stateless conversations; risky when early turns contain constraints that later turns must respect. Best deployed as a hybrid with a pinned summary header. See Rolling Window for implementation details.
2. Progressive summarization — replaces completed sub-tasks or phases with compact structured digests. Preserves conclusions while freeing token budget. Inherently lossy; must never be applied to active tool results, in-flight constraints, or partially completed tasks. See Summarization Strategy for safe and unsafe summarization patterns.
3. External storage + RAG handoff — moves older history to a database and retrieves it on demand rather than keeping it in live context. Scales indefinitely but adds retrieval latency and complexity. Retrieval may miss items, so this approach is best for reference material rather than task-critical history.
4. Prompt caching — avoids re-tokenizing a stable prefix (system prompt, tool schemas, reference documents) on every turn. Does not reduce token count or free window space; it reduces cost and latency. Complementary to compression, not a substitute.
What must never be compressed
Regardless of strategy, these items must remain verbatim in context:
- Active or in-flight tool results containing exact values (file paths, IDs, numeric outputs)
- Constraints still in force that the model must continue to respect
- Partially completed multi-step tasks where the full thread is needed to continue correctly
Upstream reduction vs. compression
Context compression handles the *dynamic* history. The other half of the budget is upstream reduction: writing tighter system prompts, limiting tool-result verbosity, and using prompt caching so stable content is not re-tokenized each turn. Both levers matter; compression alone cannot compensate for a bloated system prompt.
How It Compares
| Strategy | How it reduces tokens | Lossy? | What to compress | What to never compress |
|---|---|---|---|---|
| Rolling window eviction | Drops turns outside sliding window | Yes — early context gone | Stateless exchanges; self-contained turns | Constraints, decisions, exact tool outputs still needed |
| Progressive summarization | Replaces old turns with structured digests | Yes — detail lost, conclusions retained | Completed sub-tasks, finished research phases | Active tool results, live constraints, in-flight tasks |
| External storage + RAG | Moves history out of context entirely | Retrieval may miss items | Long-running reference material | Task-critical values that must be reliably retrieved |
| Prompt caching | Avoids re-tokenizing stable prefix | No — content unchanged | Stable system prompts, repeated tool schemas | N/A — caching doesn't remove content |
Frequently Asked Questions
When should I compress versus letting the API handle truncation?
Always prefer explicit compression. Silent API truncation removes content from the oldest turns without your knowledge, which can strip context silently — a dangerous failure mode for agentic tasks. Proactive compression lets you choose *what* is preserved.
Does prompt caching free up context window space?
No. Prompt caching reduces the cost and latency of re-reading a stable prefix but does not reduce token count or free window space. Use caching for stable overhead (system prompts, tool schemas); use compression (rolling window or summarization) for growing history.
How do I decide between rolling window and summarization?
Ask whether early turns contain information that later turns must respect. If yes — constraints, decisions, exact values — use summarization so those details are condensed rather than dropped. If each turn is self-contained and the conversation is effectively stateless, rolling window eviction is simpler and adds no latency.