Cost, Latency & Token Optimization Techniques

Optimization is a tradeoff against measured quality -- never blind. There are five standard levers for reducing cost, latency, and token usage in a Claude system.

The Five Levers

• **Prompt Caching** -- cut latency and cost on repeated stable prefixes with no accuracy loss (order stable content first, so the cacheable portion is contiguous)
• Right-size the model -- route steps to the cheapest tier that still passes the eval; use Opus/extended thinking only where the task actually needs that capability
• Trim the context -- prune stale tool output and over-fetched chunks; fewer tokens means lower cost and less context rot
• Batch -- batch latency-tolerant bulk jobs to cut cost; the Batches API is async and cheaper, but it does not reduce per-request latency
• **Cap `max_tokens`** -- bound output cost by setting an appropriate ceiling

Why These Five, and Not Others

Each lever targets a different cost driver: caching targets repeated input tokens, right-sizing targets per-token pricing, context trimming targets total input volume, batching targets throughput economics (not latency), and capping max_tokens targets output volume. Applying the wrong lever to the wrong driver -- e.g., expecting batching to speed up a single latency-sensitive request -- doesn't produce the expected saving.