Token
APIDefinition
The fundamental unit of text processing for Claude. Roughly 3-4 characters or about 0.75 words in English. Used for measuring input, output, and context window size. Costs are calculated per input and output token.
Example Usage
A 1000-word document is approximately 1300 tokens. Track token usage to manage costs.
Why It Matters for the CCA-F Exam
Token counting underpins every cost, latency, and capacity calculation on the exam. You need to know the rough sizing ratios, understand the `usage` object fields (including cache variants), and recognise that output tokens are priced differently from input tokens.
In Depth
Tokens: The Atomic Unit of Claude's Processing
Claude doesn't read text the way humans do — character by character or word by word. It processes tokens, which are chunks of text that its tokenizer has learned to recognise as meaningful units. Common short words like "the" or "is" map to a single token. Longer or rarer words may split across two or three tokens. Whitespace, punctuation, and special characters each consume tokens too.
Rough Sizing Rules
For English prose, a good mental model is:
- ~4 characters ≈ 1 token
- ~0.75 words ≈ 1 token (so 1,000 words ≈ 1,300 tokens)
Those ratios break down for other content types:
| Content type | Tokens per 1,000 chars (approx.) |
|---|---|
| English prose | ~250 |
| Minified JSON / code | ~300–400 |
| Python (well-formatted) | ~200–250 |
| URLs and file paths | ~350–500 |
| Non-Latin scripts (e.g. Chinese) | ~500–700 |
The practical takeaway: tool schemas, file paths, and JSON blobs tokenise worse than prose. Don't budget your context window based only on word count.
Where Tokens Show Up in the API
Every messages response includes a usage object:
{
"input_tokens": 1024,
"output_tokens": 312,
"cache_read_input_tokens": 800,
"cache_creation_input_tokens": 0
}input_tokens is the full count of what was sent (system prompt + messages + tool definitions). output_tokens is what Claude generated. The two cache fields appear when prompt caching is active — cache_read_input_tokens are tokens served from cache at a reduced rate.
Cost Model
Pricing is always quoted per million input tokens and per million output tokens, at different rates. Output tokens are typically priced 3–5× higher than input tokens because generating text is more compute-intensive than encoding it. For workloads where output is large (long documents, code generation), output token cost often dominates the bill. For workloads with large stable system prompts, prompt caching can cut the effective input cost substantially.
Token awareness is foundational: you cannot reason about max_tokens settings, caching strategy, or cost without it.
Example
import anthropic
client = anthropic.Anthropic()
# Count tokens before sending (avoids wasted API calls on oversized prompts)
count = client.messages.count_tokens(
model="claude-opus-4-8",
system="You are a helpful assistant.",
messages=[{"role": "user", "content": "Explain transformer attention in detail."}]
)
print(f"Estimated input tokens: {count.input_tokens}")
# Full call — inspect usage after
response = client.messages.create(
model="claude-opus-4-8",
max_tokens=1024,
system="You are a helpful assistant.",
messages=[{"role": "user", "content": "Explain transformer attention in detail."}]
)
print(f"Input: {response.usage.input_tokens}, Output: {response.usage.output_tokens}")
print(f"Cache read: {getattr(response.usage, 'cache_read_input_tokens', 0)}")Frequently Asked Questions
Why does the same text produce different token counts across models?
Each model family can use a different tokenizer vocabulary. The counts are usually close, but you should always use the API's `count_tokens` endpoint (or the `usage` response field) rather than relying on a fixed character-to-token ratio for production budget logic.
Do thinking blocks count against my input tokens on the next turn?
When thinking is enabled, the thinking content block appears in the response but is not automatically re-sent in the next request. If you include thinking blocks in your conversation history for multi-turn continuity, they do count as input tokens on that subsequent request. In practice, most architects omit thinking blocks from the history they replay unless the reasoning itself needs to be visible downstream.
Is there a free way to check token count without making a full API call?
Yes — use the `client.messages.count_tokens()` method (available in the official Python and TypeScript SDKs). It's a lightweight endpoint that returns the estimated `input_tokens` count without generating any output or incurring output-token charges. Use it in pre-flight checks before submitting requests that might exceed your context budget.