Tokens & Autoregressive Next-Token Generation

Tokens Are the Unit That Matters

Claude does not read or bill text in characters or words -- it operates on tokens, roughly word-fragments, where one Token is approximately 3-4 characters of English. Every limit you'll encounter (context window size, max_tokens, pricing) is expressed in tokens. Two prompts with the same character count can have meaningfully different Token counts depending on vocabulary, punctuation, and language.

This matters practically: you pay per input Token and per output Token, and both count against the model's context-window budget.

Autoregressive Next-Token Generation

An LLM like Claude is an autoregressive predictor. Generation works one step at a time:

1. Given everything so far (the prompt plus any tokens already generated), the model predicts a probability distribution over the next Token.
2. It samples/selects one Token from that distribution and appends it to the sequence.
3. It repeats step 1 with the extended sequence, continuing until a stop condition (a stop sequence, max_tokens, or a natural end-of-response signal) is reached.

There is no single-shot "compute the whole answer at once" step -- every Token depends on all tokens that came before it, including the ones the model itself just generated. This is why output generation is inherently sequential and why longer outputs take proportionally longer and cost proportionally more.

Common exam traps

• Assuming Token counts map cleanly to word counts or character counts -- they don't; tokenization is vocabulary-dependent and approximate (~3-4 characters per Token for English).
• Thinking of the model as retrieving or computing a full response in one step, rather than generating it Token by Token, each conditioned on all prior tokens.