SLA (Service Level Agreement)
PatternsDefinition
A commitment about a system's latency and reliability that must be grounded in what the chosen model tier, retrieval steps, and infrastructure can actually deliver, not in aspiration. A common exam trap is promising sub-second latency on a multi-step agentic pipeline with reranking -- both add latency the SLA must account for. Re-alignment is continuous: a model version, data change, or scaling need can shift what's feasible.
Tested in: CCAR-P
Why It Matters for the CCA-F Exam
CCAR-P stakeholder and solution-design items punish aspirational SLAs. If the scenario picks Opus plus multi-hop retrieval plus a human review, you cannot also promise sub-second chat.
In Depth
A service level agreement (SLA) for a Claude-powered product is a commitment about latency, availability, or quality that you can actually keep given the model tier, the number of model calls in the path, retrieval, tools, and any human-in-the-loop step. The exam trap is writing the SLA first ("p95 under 800 ms, 99.9% uptime, always-correct citations") and then picking Opus, three retrieval hops, and an evaluator-optimizer loop.
Ground the number in the path
Walk the request:
- Auth and routing
- Retrieval or MCP calls (each is a network RTT plus the tool itself)
- The Claude call (tier, prompt size,
max_tokens, whether you stream) - Optional second Claude call (structured extraction, judge, retry)
- Post-processing and the UI
Each hop has a distribution, not a single number. Streaming improves time-to-first-token, not necessarily total generation time. Batch API has no interactive latency SLA — it is the wrong backend for a user-facing chat commitment.
What to put in the SLA versus the SLO
Stakeholders hear SLA as a promise. Internally you want SLOs with error budgets. On the exam, the architect's job is to refuse a promise the architecture cannot meet, or to change the architecture (Haiku for classification, cache a large prompt, skip the second judge call on the hot path, degrade to a template when Claude is down).
Quality SLAs ("95% citation accuracy") need a held-out eval set and a measurement loop. You cannot SLA a vibe.
Availability SLAs need a defined fallback: cached answers, a smaller model, or a human queue — graceful degradation from the Context Management playbook, applied to product commitments.
See Accuracy-latency-cost tradeoff for the three-way squeeze that usually sits behind a bad SLA.
Frequently Asked Questions
Can I offer a chat SLA on top of the Message Batches API?
No. Batches are asynchronous, can take hours, and have no interactive latency target. Use the synchronous Messages API (and streaming) for user-facing chat.
Does streaming let me promise a lower total latency SLA?
Streaming improves time-to-first-token, which is what users feel, but total generation time still depends on output length, model tier, and tools. Write the SLA against the metric you can measure and keep.