Reliability Controls: Exponential Backoff, Fallback Chains & Circuit Breakers

The five metric categories above (accuracy, latency, cost, safety, security) describe a system operating under normal conditions. But a production system doesn't only face normal conditions -- dependencies rate-limit you, providers have brief outages, retrieval backends time out. RELIABILITY is the layer that decides what happens next, and it deserves to be named and tested explicitly, because a system that passes every happy-path eval can still burn its latency budget, its cost budget, and its safety posture the moment a dependency has a bad day.

Three Named Controls, Three Different Layers

Why the Layering Matters

These three controls are not interchangeable, and stacking the wrong one at the wrong layer either does nothing or actively makes an outage worse. Backoff without a circuit breaker means that during a genuine outage (not a transient blip), every single request in a busy pipeline still burns through its full retry budget before failing -- multiplying latency and cost across thousands of calls that were destined to fail anyway. A circuit breaker without a fallback chain means the system correctly stops hammering the dead dependency, but has nothing to serve instead, so the feature simply goes dark for the cooldown window rather than degrading gracefully.

A Worked Example: An OCR Pipeline Under Provider Strain

Picture a document-processing pipeline that calls a third-party OCR API for every incoming scan. Under normal load it works fine. Then the OCR provider starts intermittently returning 429s under its own load spike.

• With no reliability controls at all: every 429 is a failed job. Customers see failures proportional to exactly how strained the provider is at that instant -- noisy, unpredictable, and entirely outside your control.
• With exponential backoff only: most transient 429s now succeed on the second or third attempt, a real improvement. But if the provider goes fully down for ten minutes (not just strained), every request arriving during that window still burns through its full retry schedule before eventually failing -- multiplying end-to-end latency across every job in the queue and multiplying the API cost of calls that were never going to succeed.
• Adding a circuit breaker at the orchestration layer: after some threshold of consecutive failures (say, five in a row), the breaker trips open and the pipeline stops calling that provider entirely for a cooldown period -- no more wasted retries, no more compounding latency.
• Adding a fallback chain: while the breaker is open, requests route to a secondary OCR provider (slower, slightly less accurate, but functioning) or serve a cached extraction for documents seen recently, so the feature degrades instead of going dark.

Together, the three controls turn "the OCR provider is having a bad ten minutes" from a customer-visible outage into a brief, mostly invisible dip in quality -- exactly the kind of resilience an eval built only from happy-path examples will never surface, because a happy-path eval never simulates the dependency actually failing.

Connecting Back to the Five Dimensions

Reliability failures don't show up as their own metric category in 4.1.2 -- they show up as symptoms *within* the five dimensions you already track: a retry storm shows up as a cost anomaly, waiting through a full retry schedule shows up as a latency SLA breach, and a fallback chain that silently routes to a less-trusted or less-capable path can quietly erode accuracy or even security if the fallback wasn't designed with the same guardrails as the primary path. That's precisely why reliability has to be tested and monitored deliberately, rather than assumed as a side effect of the primary system being well-built.

Common exam traps

• Naive retry mistaken for resilience. A fixed-interval or immediate retry loop against a struggling dependency is not a reliability control -- it's a self-inflicted amplifier. A burst of immediate retries against a dependency that's already failing under load looks, from the dependency's side, indistinguishable from a denial-of-service attack. Exponential backoff (increasing delay between attempts), not naive immediate retry, is the correct default.
• Treating the three controls as interchangeable. Backoff, fallback chains, and circuit breakers solve different problems at different layers; a scenario describing repeated failed retries against a dependency that's fully down is asking for a circuit breaker (stop trying), not more aggressive backoff (which still eventually tries and still eventually fails).