2.1 Model Selection and Tradeoffs

Picture a hiring manager who insists every task in the company — from photocopying to closing the biggest deal of the year — goes to the single most senior person on staff. It sounds like a way to guarantee quality. In practice it's a way to guarantee a bottleneck: the senior person is expensive, in high demand, and slower to free up than a junior hire who could photocopy just as well. Claude gives you the same choice a manager has, except the "staff" are model tiers, and the exam wants you to staff each task correctly rather than reflexively hiring the most senior name on the roster.

Claude ships in three tiers that trade capability for speed and cost. Haiku is the fastest and cheapest — built for high-volume, well-defined, latency-sensitive work like classifying a support ticket or routing a request to the right queue. Sonnet is the balanced workhorse: capable enough for most production reasoning, priced and paced for everyday use. Opus is the most capable tier, reserved for the hardest reasoning, planning, and orchestration steps — the ones where getting it right matters more than getting it fast or cheap. None of the three is "the good one" in isolation; each is the right one for a particular job.

Every model choice sits inside a triangle of quality, latency, and cost. Push toward more quality and you typically pay in latency, cost, or both. Push toward less latency and cost, and quality has less room to work with. There is no point at the center of that triangle that is objectively "best" — the triangle only becomes a decision once you pin it against a real requirement: a latency SLA of 400ms, a budget of $0.002 per request, or a quality bar that a human reviewer would actually accept.

This is the discipline the exam is really testing: not "which model is the best model," but "which model resolves this specific constraint." A real-time chat widget answering simple FAQs has a tight latency budget and low quality ceiling on the questions it's likely to see — Haiku fits. A one-time legal-contract risk analysis has almost no latency pressure and an extremely high cost of being wrong — Opus, quite possibly with extended thinking, fits. The same organization might correctly use both models in the same week, even in the same pipeline, because the triangle looks different for each task.

The triangle applies per step, not per system — and that distinction is where a lot of architects go wrong. A production pipeline is rarely one uniform task; it's a sequence of very different steps chained together, and each step has its own difficulty and its own budget. Treating "which model should our pipeline use" as a single, system-wide decision throws away the biggest lever you have: assigning cheap steps to a cheap tier and hard steps to a capable one.

This is exactly the coordinator-and-specialists pattern from multi-agent orchestration, applied to model choice instead of just task delegation. A document-processing pipeline might use Haiku to classify each incoming document type (cheap, high-volume, simple), Sonnet to extract and summarize the content (moderate reasoning, still frequent), and Opus only for the final step that synthesizes findings across documents into a decision memo (rare, hard, expensive to get wrong). Three tiers, one pipeline, each doing the job it's actually suited for.

So far we've picked a tier. Extended (adaptive) thinking is a separate lever layered on top: it lets Claude reason step by step before producing its final answer, and that reasoning is billed like output — more tokens, more latency — in exchange for better performance on genuinely hard reasoning tasks. It's not a quality setting you flip on for everything; it's a lever you pull only where the task's difficulty justifies the price.

Think about how a careful professional actually works. For a routine email, they type the reply and send it — there's no value in sitting and deliberating. For a decision with real stakes — should we restructure this contract clause, does this financial model hold up — they slow down and reason it through on purpose, because the extra time buys a better outcome. Extended thinking is that same deliberate slowdown, applied selectively. Turning it on for a simple classification step doesn't make the classification more correct; it just makes it slower and more expensive for no gain.

The previous note described extended thinking configured with thinking={"type": "enabled", "budget_tokens": N} — a fixed token ceiling you hand-pick before the request goes out. That shape is not the API's end state. Newer model generations are moving to a qualitative EFFORT parameter, and an architect who only knows the budget_tokens shape will eventually write a request that a newer model rejects outright, not just one that performs sub-optimally.

The difference is a real design shift, not a rename. budget_tokens asks you to predict token consumption in advance — guess how many tokens the model will need to reason through a task, and you're either right (the budget covers it) or wrong (truncation mid-thought, or wasted headroom). effort asks you to state how much depth the task deserves — using levels such as low, medium, and high — and lets the model manage its own token spend within that envelope for the specific input in front of it. That's a judgment an architect is well-positioned to make ("this synthesis step matters, spend what it needs") replacing a guess an architect is poorly positioned to make ("this synthesis step will need exactly 6,000 tokens of reasoning").

The deprecation timeline matters because it's sharp, not gradual. On the model generation where budget_tokens is deprecated, it still works — with a warning you'd only notice if you were watching for it. On the generation after that, passing budget_tokens returns an outright error, and the request never reaches the model at all. A system that pins an older model version specifically to keep budget_tokens working is accumulating a debt that becomes a hard migration requirement the moment that pinned version is retired — not a refactor you can defer indefinitely.

This produces the single most avoidable incident in this area: an architect treats a model-version bump as routine maintenance, doesn't check whether the new generation has moved past budget_tokens, and ships a request shape the new model rejects outright — turning a routine upgrade into an unplanned outage. Because model choice is already a per-step decision in a mixed-model pipeline (2.1.3), the same pipeline can legitimately need budget_tokens on one step (an older, still-supported model) and effort on another (a newer model) at the same time — audit each step's thinking configuration independently rather than assuming one setting applies system-wide.

Model-tier selection and extended-thinking discipline both cut cost by changing HOW MUCH reasoning a request pays for. The Batches API cuts cost through a completely different mechanism: it changes WHEN and HOW a request is processed, trading immediacy for a substantial price reduction on the exact same underlying work. It belongs in the same toolkit as tier selection and extended thinking, but it solves a different problem, and conflating the two is a specific, well-tested exam trap.

The Batches API accepts many independent requests submitted together, processes them asynchronously — not necessarily immediately, not necessarily in submission order — and returns results once the batch completes, typically well after the submission call itself has already returned. In exchange for accepting that asynchrony, batched requests are billed at roughly 50% of standard API pricing for the same tokens. That makes it the right tool specifically for large volumes of work where nobody is waiting on any single result in real time: reprocessing a document archive overnight, re-scoring a historical dataset against an updated eval, generating summaries for thousands of support tickets as a nightly job. Batch size is illustratively on the order of tens of thousands of requests per submission rather than a handful — the architect-relevant point is the order of magnitude (this is built for genuinely bulk workloads), and the exact current numeric ceiling should be verified against live API documentation rather than memorized as a fixed number.

Here is the distinction the exam leans on hardest, and it is worth stating as plainly as possible: the Batches API reduces COST. It does NOT reduce PER-REQUEST LATENCY. It does not make any individual request come back faster — there is structurally no streamed, real-time response for a batched request, because the whole mechanism depends on the caller being willing to submit now and collect results later. A scenario describing a user waiting in real time for a response is never solved by batching, no matter how much cheaper it is per token, and no matter how tempting "but it's 50% cheaper" sounds as an argument. Reaching for the Batches API to speed up an interactive chat widget is a throughput optimization applied to a latency problem — the exact same category of mismatch as reaching for the biggest model when the business actually asked for cost control.

There's a final trap in model selection that has nothing to do with picking the right tier: what happens when the model itself changes under you. A new model release can shift behavior — accuracy on your exact task, adherence to your output format, tone — even when nothing about the API contract changed and even when the new version is, on paper, strictly more capable. A prompt you tuned carefully against one version is not guaranteed to produce the same quality on the next.

The architectural discipline here mirrors how you'd treat any other dependency upgrade: you don't let production silently float to "whatever the newest model is" and hope nothing breaks. You pin the specific model version your system runs against, and when a new release is available, you treat adopting it as a change that requires the same eval gate a prompt edit would (this is the direct link forward to Domain 4's evaluation discipline) — re-run your evaluation set against the new version before promoting it, rather than assuming newer automatically means better for your specific prompt and task.

• 1.Pin the exact model version in production rather than floating to "latest."
• 2.When a new release appears, re-run your evaluation set against the pinned prompt paired with the new model.
• 3.Promote deliberately — only after the eval results confirm quality held or improved, not on the assumption that newer implies better.