3.5 Progressive Discovery, Observability & Accuracy-Latency Tradeoffs

Suppose your integration now has 200 MCP tools across several servers, a large internal documentation set, and dozens of schemas describing downstream services. There are two fundamentally different ways to make an agent aware of all of that, and the choice between them determines whether the integration scales gracefully or collapses under its own weight as it grows.

Monolithic context front-loads everything — every tool definition, every schema, every doc — into the context window up front, on every single request, regardless of whether the current task touches 2 of those 200 tools or all of them. It's the simplest thing to implement: no discovery logic, nothing to fetch on demand, just hand the model the whole map before it takes a single step. And that simplicity is exactly why it's tempting to default to — until the map gets big enough that carrying it everywhere costs more than it helps.

Progressive discovery inverts this: expose a LEAN surface up front — maybe just "list available tools" and "list available resources" — and let the agent fetch DETAIL on demand as the task actually requires it. List resources, then read only the one you need; discover which tools exist, then pull the full schema only for the one you're about to call. The agent pays the token cost of detail only for what it actually touches on a given turn, not for the entire catalog whether or not it's relevant.

Don't over-correct into treating progressive discovery as universally superior — the exam tests whether you know when monolithic context is actually FINE, not just when it's bad. For large integrations, progressive discovery is the scalable pattern: it lets an agent operate against hundreds of tools or a large resource catalog without paying the context cost of loading all of it on every turn. But monolithic context is acceptable, even preferable, when the full set is small and stable — a handful of tools that rarely change costs almost nothing to load up front, and the discovery round-trips progressive discovery requires (list, then fetch detail) add latency and complexity that buys you nothing when there was never much to discover in the first place.

So the actual test isn't "discovery good, monolithic bad" — it's matching the pattern to the SIZE and STABILITY of what's being exposed, exactly the same discipline as matching retrieval strategy to data shape in Lesson 3.3. A ten-tool agent that rarely changes its tool set: load it all, don't bother with discovery machinery. A two-hundred-tool integration spanning multiple MCP servers, some added or updated independently: progressive discovery, every time.

A well-designed integration that you can't SEE inside is still a liability — you find out something's wrong when a user complains, not when a dashboard flags a regression. Production integrations need to log and trace enough to reconstruct what happened, after the fact, without guessing. Five things belong in that trace, at minimum: request/response pairs, tool invocations and their arguments, retrieval hits (which chunks came back for which query), stop_reason (recall Domain 1 — the deterministic signal for why a turn ended), and per-hop token usage.

Logging isolated events isn't enough for multi-step agent runs — you need to TRACE the whole run so that when something goes wrong, you can walk the failure back to its FIRST deviation, not just the step where it became visible. This mirrors a diagnostic habit that shows up across the exam: the earliest point where behavior diverged from expected is usually the actual root cause, and the step where a user first noticed something wrong is often several steps downstream of where the real problem started.

• •Request/response pairs — the raw exchange with the model, for reconstructing exactly what was asked and answered.
• •Tool invocations and arguments — which tools were called, with what inputs, so a wrong result can be traced to a wrong call.
• •Retrieval hits — which chunks came back for a given query, essential for diagnosing RAG regressions (more on this below).
• •stop_reason — why each turn ended, the same deterministic signal from Domain 1's agentic loop.
• •Per-hop token usage — cost and context consumption at each step of a multi-hop run.

At scale, logging every single request in full detail everywhere becomes its own cost and noise problem, so production observability layers in four further practices: sampling (inspect a representative subset of traffic rather than everything), structured logs (machine-parseable records rather than free text you'd have to grep by hand), latency/error dashboards (surfacing drift in near real time), and per-domain quality metrics (catching a regression specific to one integration surface — say, a RAG pipeline's answer quality — before it shows up as a generic complaint).

Here's the diagnostic scenario the exam returns to for this domain, and it rewards exactly the observability discipline from 3.5.3. A RAG system suddenly starts returning confident-SOUNDING but factually incorrect answers, right after a document refresh, while latency and the model version are both unchanged. Where do you look first?

Walk the timeline. Nothing about the MODEL changed — same version, same latency profile, so the generation step itself is an unlikely culprit. What DID change, at precisely the moment the symptom appeared, is the underlying document set — via the refresh. That correlation is the whole diagnosis: the retrieval/indexing step is the first place to investigate, because a broken re-index or mismatched embeddings will produce exactly this symptom — the model generating a fluent, confident answer from chunks that are stale, irrelevant, or simply wrong, with nothing about the model's own behavior having changed at all. The model isn't lying; it's answering faithfully from bad material it was fed.

This is precisely why logging retrieval hits (3.5.3) matters so much in practice — without a record of WHICH chunks were retrieved for a given failing query, you're left guessing between a model problem and a retrieval problem, when the timing correlation with the refresh already told you which one it almost certainly is.

Every lever an architect can pull in an integration touches at least two of accuracy, latency, and cost — usually trading one for another. The job isn't to eliminate the tradeoff (you can't) but to name which constraint dominates for THIS integration and justify the configuration against it explicitly, rather than defaulting to whichever knob feels safest.

Notice that prompt caching is the odd one out on this list, and it's worth understanding why: caching doesn't change WHAT gets sent to the model or what the model generates from it — it changes how much you pay and how long you wait to send the same repeated content again. That's why it's a free win rather than a tradeoff, and why it showed up earlier as the thing that made contextual retrieval's per-chunk context generation affordable (Lesson 3.3) — the same mechanism, the same kind of "cost and latency down, accuracy untouched" benefit, in two different places in this domain.

For every other lever, the discipline is the same: state which constraint — accuracy, latency, or cost — actually dominates for this integration's real requirement, and justify the configuration against THAT, not against a generic preference for speed or precision. A support-triage integration with a strict SLA might reasonably accept a small accuracy hit from retrieving fewer chunks, because the SLA is the binding constraint. A compliance-review integration might accept the added latency of a reranking step, because there, accuracy is non-negotiable and the SLA has more slack. Neither choice is universally "more correct" — each is correct relative to its own stated requirement.

You now know when to expose a lean surface versus front-load everything, what to log and trace so failures are diagnosable rather than mysterious, how to run the "confident but wrong after a refresh" diagnostic, and how to justify an accuracy-latency-cost knob against a stated requirement rather than a gut preference. This closes out Domain 3 — from right-sizing tools, through secure identity, through retrieval design, through protocol choice, to running the whole thing observably in production.

That completes Domain 3. You can now evaluate an integration end to end: is the tool set right-sized, is access correctly scoped to real identities, is retrieval well-designed for the data's actual shape, is the connection mechanism matched to its reuse profile, and is the whole thing observable and justified against its real accuracy-latency-cost requirement.