Token Pricing, Cost Modeling & the Usage Field
CoreModel token cost and track usage · Difficulty 1/5
Explanation
Pricing Is Per-Token and Asymmetric
Claude usage is billed per token, but not at one flat rate: input and output tokens are priced differently, with output tokens typically more expensive than input tokens. There are also separate, distinct rates for cache writes and cache reads (covered further under prompt caching).
Cost Modeling
Modeling expected cost for an application means: estimate the token volume for each request type (input tokens per call, expected output tokens per call, any cached content) and multiply by the applicable per-token price, summed across expected traffic volume. This is the basic arithmetic behind any "what will this cost at scale" question.
The `usage` Field
Every API response includes a usage field reporting the actual input, output, and cache token counts for that call. Instrumenting this field is how a team:
- Tracks real, observed cost (rather than only modeled/estimated cost)
- Detects context bloat or unexpectedly large requests before they become a recurring cost problem
- Feeds a dashboard or alert for spend monitoring
Capping `max_tokens`
Setting `max_tokens` to a realistic ceiling for the expected output length is a direct lever against runaway generation cost -- without a sensible cap, a malfunctioning prompt or an edge-case input could generate far more output (and cost) than intended.
Common exam traps
- Assuming input and output tokens are priced the same -- output is typically the more expensive side of the pair.
- Treating cost modeling as a one-time estimate rather than something the
usagefield lets you continuously verify against actual traffic.
Key Takeaways
- Input and output tokens are priced differently; output tokens are typically more expensive than input tokens
- Cache writes and cache reads have their own distinct rates within the overall pricing structure
- Cost modeling = estimated tokens per request type x per-token price, summed across expected traffic
- The usage field (input/output/cache tokens) on every response is the instrumentation for tracking real cost and catching bloat
- Capping max_tokens to a realistic ceiling prevents paying for runaway generations
Glossary Terms
The recurring architectural tension where an integration decision that improves accuracy (e.g., reranking, retrieving more chunks) typically adds latency and cost, and vice versa. The architect's job is not to eliminate the tradeoff but to make it explicit and justify the chosen configuration against whichever constraint the stated requirement names as dominant. Prompt caching a stable repeated context is the rare exception that improves cost and latency with no accuracy loss.
API parameter that sets the maximum number of tokens Claude will generate in a single response. If generation would exceed this limit it is truncated and `stop_reason` is set to `"max_tokens"`. This is a required parameter — omitting it returns a 400 error.
Related Concepts
Prompt Caching & Cache Checkpointing
A cache_control field of type "ephemeral" on a content block marks a breakpoint; up to 4 breakpoints are allowed per request
The Context Window as a Shared Token Budget
The context window is the max tokens (input + output combined) a model can consider in one request