7.2 AI-Assisted Workflows, Automation & Operational Debugging
7.2.1 Why "Faster" Isn't the Same as "Better"
Once a team's Claude Code is configured consistently (Lesson 7.1), the obvious next question is: how do you actually get MORE done with it? The tempting, wrong answer is to remove friction — skip the code review, load the whole repo into context so Claude "just knows everything," let an agent run against production with no one watching. Every one of those moves does make things go faster, for exactly as long as it takes for the first mistake to slip through unnoticed. Then it gets slower, because now someone has to find the mistake, understand what actually happened, and undo it — usually with less information than they'd have had if the mistake had been caught immediately.
The architect's actual job in this task statement is subtler than "make it faster." It's making the SPEEDUP durable — building workflows where throughput goes up and the checks that catch mistakes stay firmly in place. Anthropic's own best-practice guidance for Claude Code names this directly: leverage with guardrails. The rest of this lesson is really one idea, applied to four different situations: a single interactive session, an automated pipeline, a custom internal agent, and — when something inevitably still goes wrong — the debugging process itself.
The one idea to hold onto
Productivity gains from AI-assisted tooling come from a disciplined loop and lean context, not from removing checks. Every mechanism in this lesson -- the interactive loop, headless automation, the Agent SDK, and debugging -- is that same discipline applied at a different scale.
7.2.2 The Gather-Context, Plan, Act, Verify Loop
Start with the smallest unit: one developer, one session, one task. Anthropic's best-practice workflow for this is a four-stage loop, and each stage exists to prevent a specific, predictable way the previous stages go wrong.
GATHER CONTEXT means giving the agent exactly the information it needs to do the task — the relevant files, the relevant conventions, the actual error message — and no more. This stage has a trap on both sides. Too little context and the agent guesses, filling gaps with plausible-sounding assumptions that happen to be wrong. Too much context — dumping in the entire codebase "just in case" — causes something specific enough to have its own name: CONTEXT ROT, where an overloaded or stale context window measurably degrades output quality, even though nothing in it is technically incorrect. PLAN means letting the agent lay out its intended approach BEFORE it touches any code, so you (or a reviewer) can catch a wrong-headed strategy while it's still a paragraph of text, not a diff across fifteen files. ACT is the agent actually making the change, against the plan you've now had a chance to see. VERIFY is checking the result — running tests, running the build, having a human or a second agent review the diff — before treating the work as done.
Gather-context, plan, act, verify: four stages, each one a check on the stage before it. Skip verify and the whole loop degrades into 'hope it worked.'
| Stage | What happens | Failure it prevents |
|---|---|---|
| Gather context | Give the agent the right, lean information | Guessing (too little) or context rot (too much) |
| Plan | Agent lays out its approach first | A wrong strategy discovered only after the code is already written |
| Act | Agent executes the plan | Drift between the agreed approach and what actually got built |
| Verify | Tests, build, human or agent review | A broken or subtly wrong result being treated as done |
The loop is a chain of checks, not a single step. Removing any one stage removes the specific failure that stage was catching.
7.2.2 — Key Concept
The best-practice AI-assisted workflow is gather-context -> plan -> act -> verify, with LEAN context at every stage to avoid context rot. Productivity comes from the discipline of all four stages together, not from any single fast step.
7.2.3 Scaling the Loop: Agent SDK, Headless Mode, and Custom Commands
The four-stage loop above describes one developer in one interactive session. But an architect's real leverage comes from making that same disciplined loop run without a developer sitting there at all — across many repos, on a schedule, or triggered automatically by a pull request. Three mechanisms make that possible, and they compose.
The AGENT SDK lets a team build custom internal agents and automate workflows programmatically — going beyond the interactive tool entirely. This is how an architect turns "a developer manually asks Claude Code to check for outdated dependencies" into "a scheduled agent checks every repo nightly and files a report," or automates codebase modernization and other repetitive engineering tasks that would otherwise eat a developer's whole afternoon, one repo at a time. HEADLESS (non-interactive) mode is what makes any of this possible inside a pipeline: Claude Code can run without a human at the keyboard, which is exactly what a CI job or a cron trigger needs — there's no one there to type a response if it waits for input. And CUSTOM SLASH COMMANDS (including GitHub integration) standardize the common, repeated team tasks — turning "how we always start a PR review" from a habit one developer remembers into a single, consistent, team-wide invocation anyone can trigger the same way.
Notice how these three stack on top of each other rather than replacing the loop from 7.2.2: a custom internal agent (built with the Agent SDK) runs in headless mode (so it fits inside CI), triggered by a standardized custom command (so every developer invokes it the same way). The discipline doesn't vanish when you automate it — the gather-context, plan, act, verify structure is exactly what gets ENCODED into the automation. An agent that automates code review without a verify step isn't more productive; it's just a faster way to ship undetected mistakes.
| Mechanism | What it adds | Typical use |
|---|---|---|
| Agent SDK | Custom, programmatic internal agents | CI/CD checks, codebase modernization, repetitive engineering tasks |
| Headless mode | Non-interactive execution, no human at the keyboard | Running inside scripts, CI pipelines, scheduled jobs |
| Custom slash commands / GitHub integration | A single, consistent, team-wide invocation of a common task | Standardizing PR review kickoff, release prep, etc. |
The three mechanisms compose: an Agent-SDK-built agent, running headlessly, triggered by a standardized command.
7.2.3 — Key Concept
The Agent SDK builds custom internal agents; headless/non-interactive mode lets Claude Code run inside CI and scripts with no human present; custom slash commands and GitHub integration standardize common tasks. All three extend the interactive verify loop into automation -- they don't replace its discipline.
Everything above can be misapplied in the same single way, which is worth calling out directly because it is exactly how the exam frames this task statement's wrong answers. The trap is equating "AI-assisted" with "unsupervised." An exam scenario describing an agent running fully unattended against production, with no verify step and no human review anywhere in the pipeline, and calling that a productivity win, is describing the anti-pattern — not a best practice, no matter how much faster it sounds. The corrective is always the same: the speedup is real, but it comes FROM the discipline, not from removing it. A headless agent that skips verify isn't a more efficient version of the loop — it's a different, unsafe process that happens to look similar.
7.2.3 — Exam Trap
Equating 'AI-assisted' with 'unsupervised' is the domain's central misconception. Reject any answer that removes tests or review to go faster, runs an agent fully unsupervised against production, or maximizes context 'just in case.' The correct answer preserves gather-context -> plan -> act -> verify, with lean context, even inside full automation.
7.2.4 When Something Still Breaks: Isolating the Layer
No amount of discipline eliminates every failure — it just makes failures rarer and easier to diagnose when they happen. That's the last piece of this domain: Task Statement 7.3, supporting debugging and operational issue resolution once a Claude-powered system has already misbehaved. And the very first move, before touching anything, is to resist the urge to fix the first thing that looks wrong and instead isolate WHICH LAYER actually failed.
A Claude-powered system has (at minimum) four layers a failure can originate in, and which one is broken decides what the fix actually is. TRANSPORT failures are HTTP or auth errors — the request never even reached the model correctly. INTEGRATION/PARSING failures happen when the model responded successfully (a clean 200) but YOUR code mishandled that response afterward. RETRIEVAL failures mean a RAG pipeline handed the model stale or irrelevant chunks — the model reasoned perfectly well over the wrong material. And MODEL OUTPUT failures are the trickiest to sit with: the response is well-formed, fluent, and confidently wrong — nothing upstream broke, the model itself reasoned to the wrong answer.
| Layer | What's actually broken | Where to look |
|---|---|---|
| Transport | The request/response exchange itself | HTTP status, auth errors, network failures |
| Integration / parsing | Your code mishandling a successful response | Response-handling code, downstream parsing logic |
| Retrieval | The context fed to the model was wrong | RAG pipeline, retrieved chunks, ranking |
| Model output | The model reasoned to a wrong (but fluent) answer | Prompt, task framing, examples |
Four layers, four different fixes. A well-formed but wrong answer (model output) and a mishandled valid response (integration) look similar from the outside but need opposite fixes.
The exam's signature version of this trap is exactly what it sounds like: patching the prompt when the actual bug lives in the integration code that mishandles a perfectly fine response (or the reverse — rewriting parsing logic to work around what's really a prompting problem). Neither fix touches the real defect. Worse, a prompt rewrite that happens to mask an integration bug can make the system look fixed while leaving the underlying mishandling in place, ready to resurface the next time the model's phrasing shifts even slightly.
7.2.4 — Key Concept
Isolate a failure to its originating layer -- transport, integration/parsing code, retrieval, or model output -- BEFORE changing anything. Patching the wrong layer (prompt vs. integration code, most commonly) fixes nothing and can mask the real defect.
7.2.5 Walking the Trace Back to the Earliest Deviation
Knowing the four layers exist doesn't tell you which one actually failed in front of you — for that you need to look at what actually happened, in order. The discipline is: walk request/response pairs, tool calls, retrieval hits, stop_reason, and token usage back to the EARLIEST deviation, not just the final symptom. The visible failure — a bad answer on screen, an error banner — is usually several steps downstream of where things actually went wrong, and fixing at the symptom instead of the origin is how the same bug survives a dozen well-intentioned patches.
Picture a support agent that eventually gives a confidently wrong refund policy. The final symptom is the wrong sentence in the reply. Walking backward: the model's response was fluent (so, at first glance, this looks like a model-output problem) — but the retrieval hits it was given were for a DIFFERENT product's refund policy, because a rewrite of the search query upstream dropped the product filter. The earliest deviation isn't in the model's reasoning at all; it's in the retrieval layer, several steps before the model ever saw anything. Fix the retrieval query, and the "model output problem" disappears on its own — because it was never a model problem to begin with.
Alongside tracing backward, an architect checks a short list of OPERATIONAL BASICS before assuming a deeper defect at all, because these three explain a large share of real incidents and each has a trivial fix once identified: truncated output almost always means stop_reason: max_tokens, and the fix is raising max_tokens — not rewriting the prompt, and certainly not treating it as a security or compliance issue. A MODEL VERSION MISMATCH — the deployed model isn't the one that was tested and tuned against — can silently change behavior in ways that look like a regression but are really a configuration drift. And CONTEXT BLOAT late in a long session degrades output quality on its own, independent of any prompt or code defect, which is the same context-rot idea from 7.2.2 showing up now as a live incident instead of a design principle.
# Walking a trace back to the earliest deviation, not the final symptom
for step in trace.steps:
if step.type == "retrieval" and step.chunks_relevance_score < THRESHOLD:
# earliest deviation found -- stale/irrelevant chunks fed downstream
return RootCause(layer="retrieval", detail=step)
if step.type == "model_response" and step.stop_reason == "max_tokens":
# operational basic: truncation, not a quality or security issue
return RootCause(layer="operational", fix="raise max_tokens")
if step.type == "model_response" and step.model_version != EXPECTED_MODEL:
return RootCause(layer="operational", fix="model version mismatch")
# only if none of the above explain it, treat the model's reasoning itself as the cause
return RootCause(layer="model_output", detail=trace.final_response)7.2.5 — Exam Trap
A truncated or cut-off response points to stop_reason: max_tokens -- raise max_tokens. It is NOT evidence of prompt injection, a compliance/HIPAA violation, or vague 'capability bloat.' Reject any answer that treats an operational limit as a security or quality defect.
Finding and fixing the real root cause is necessary, but it's not the last step. If the same class of failure can recur — a retrieval query that drops a filter under certain phrasings, a response type your parsing code doesn't handle — and the only thing standing between it and the next incident is a human noticing again, you haven't actually closed the loop. The last move is feeding the confirmed fix back into evals and monitoring, so the exact same issue is caught automatically the next time it happens, rather than relying on the same manual diagnosis to repeat. This is where Task Statement 7.3 connects directly back to the evaluation and monitoring discipline from earlier in the certification: a debugging session that ends with a new eval case covering the exact failure mode, or a monitoring alert tuned to catch the same signature earlier, has actually made the system more reliable going forward — not just fixed for today.
Where this shows up on the exam
7.2/7.3 questions almost always describe either (a) an AI-assisted workflow that skipped verification and call that 'productivity' -- reject it and pick the answer that restores gather-context/plan/act/verify -- or (b) a misbehaving system where the fix is patching the wrong layer, or missing an operational basic like stop_reason: max_tokens. Isolate the layer, walk the trace to the earliest deviation, check the operational basics first, and feed confirmed fixes back into evals and monitoring.
7.2.6 Runbooks & Escalation Paths: Making Every Diagnosis Reusable
Layer isolation and trace analysis, above, describe how to diagnose a SINGLE incident well. But an architect who wants a team to be operationally self-sufficient -- not dependent on the one person who happens to remember how a similar issue was solved six months ago -- needs to turn each diagnosis into a durable, reusable artifact. Two named deliverables do this: the RUNBOOK and the ESCALATION PATH. Neither is a new diagnostic technique; both are what you build FROM every diagnosis you already know how to do.
A RUNBOOK is a living document mapping SYMPTOM -> LIKELY ARCHITECTURE CAUSE -> FIRST DIAGNOSTIC ACTION, built up from real incidents as they happen. The first time a given failure pattern occurs, someone has to do the full trace-analysis, layer-isolation investigation from scratch -- that's unavoidable, and it can eat hours. The point of a runbook is that the SECOND occurrence of that same pattern shouldn't cost hours again; it should be a five-minute lookup: recognize the symptom, check the runbook, take the first action it names, and only fall back to a from-scratch investigation if the known cause doesn't actually match this time.
| Symptom | Likely cause | First action |
|---|---|---|
| Sudden latency spike with no traffic change | Provider-side model version rollout | Check stop_reason and response headers for a model-version change; compare against the pinned version |
| Retrieval results all subtly off-topic, starting at a specific timestamp | A scheduled re-index job ran with a stale embedding config | Check the re-index job's logs and config version against the last known-good run |
| A specific tool call starts silently returning empty results, no error thrown | Downstream API changed its response shape without a version bump | Diff a fresh raw response against a captured known-good sample from before the symptom started |
| Cost per request doubles overnight with no code or prompt change | A scheduled job or retry loop is looping against a failing dependency | Check recent request volume per endpoint for an unexplained spike, and check for an open circuit-breaker condition |
A short example runbook. Each row names a specific architectural mechanism, not a restated symptom -- that's what makes the first action actually checkable in five minutes.
Notice the shape of every row: the symptom is something anyone on call would actually notice without special expertise, the cause names a specific architectural layer or mechanism -- not "something's wrong" -- and the first action is a concrete, checkable step, not "investigate further," which isn't actionable at all. A runbook entry that just restates the symptom back as the "cause" ("Symptom: latency is high. Cause: latency is high.") saves no time on the second occurrence, because it doesn't point anywhere specific to look.
An ESCALATION PATH is a defined, documented chain -- for example, on-call engineer -> platform team -> vendor support -- for issues that exceed what the runbook covers. Its entire purpose is to remove guessing about who to page during an actual incident, when guessing costs real minutes and real stress. Without a documented path, an on-call engineer facing a genuinely novel failure has to improvise who to contact, often by asking around a Slack channel and hoping someone senior happens to be online -- exactly the kind of ad hoc, person-dependent process the rest of this domain has been arguing against since 7.1's discussion of tribal knowledge trapped in one person's head. A usable escalation path names, for each step, WHO (a role, not a specific person who might be on vacation), WHEN to escalate to that step (what specifically the previous step couldn't resolve), and WHAT information travels with the escalation, so the next person isn't starting from zero. "Escalate to the platform team if the runbook's first action doesn't identify the cause within 15 minutes, and hand them the incident timeline plus whatever traces have already been pulled" is a usable criterion; "ask someone if you're stuck" is not.
- •The runbook and escalation path work together as a triage funnel: check the runbook first -- fast, cheap, self-service
- •If the symptom doesn't match a known entry, or the known first action doesn't resolve it, escalate along the defined path rather than re-deriving a diagnosis from scratch
- •Escalating immediately for every issue, regardless of whether it's actually novel, burns the time of increasingly senior (and increasingly scarce) people on things a runbook entry would have handled in five minutes
- •Both artifacts get built up incrementally -- every new incident that isn't already in the runbook is a candidate for a new row once it's diagnosed
7.2.6 — Key Concept
A runbook maps symptom -> likely architecture cause -> first diagnostic action, built from real incidents, so the second occurrence of a known issue is a five-minute lookup instead of a from-scratch investigation. An escalation path is a defined, role-based chain (on-call -> platform team -> vendor support) for issues the runbook doesn't cover, removing guesswork about who to page. Both are what you build FROM every diagnosis, not a new diagnostic technique.
7.2.6 — Exam Trap
A scenario describing an on-call engineer having to guess who to contact, or wait for someone to notice a Slack message, is describing the ABSENCE of an escalation path -- the fix is a documented, role-based chain with clear escalation criteria, not "communicate better" in the abstract. Similarly, a runbook entry that only restates the symptom as the cause, with no specific mechanism named and no concrete first action, doesn't count as a real runbook entry -- watch for that distractor when a scenario claims a team "has a runbook" that doesn't actually behave like one.
7.2.7 Put It Together: Build and Break a Small Pipeline
You now have the whole domain: configuring a team's tools and environments (7.1), and here in 7.2, the disciplined loop that makes AI-assisted work actually productive, the automation mechanisms that scale it, the debugging discipline for when something still goes wrong, and the runbook/escalation-path artifacts that turn every diagnosis into something the whole team can reuse instead of relying on one person's memory. As always, the fastest way to make the isolation habit automatic is to practice it on a deliberately broken example.
7.2.7 — Build Exercise (40 min)
(1) Take a small coding task and run it through gather-context -> plan -> act -> verify explicitly, writing down what you included in context and why, and what verify step you ran at the end. (2) Wrap the same task in a headless invocation (claude -p or equivalent non-interactive mode) so it could run inside CI with no one watching -- and confirm your verify step still runs automatically rather than being silently dropped. (3) Take (or construct) a scenario with a truncated response and confirm it's stop_reason: max_tokens, not a security issue. (4) Take a scenario where a response is well-formed but wrong, and walk the trace backward through retrieval, tool calls, and stop_reason before concluding it's actually a model-output problem rather than a retrieval or integration bug hiding upstream. (5) Write down what eval case or monitoring alert you'd add so the same failure is caught automatically next time.
That closes Domain 7 -- the smallest domain by exam weight, but the one that determines whether every architectural decision from the other six domains actually reaches a team's daily practice, and whether the systems those decisions describe stay healthy once they're running in production.
Key Takeaways
- ✓The best-practice AI-assisted workflow is gather-context -> plan -> act -> verify, with LEAN context at every stage to avoid context rot; skipping any stage removes the specific failure that stage exists to catch.
- ✓Equating 'AI-assisted' with 'UNSUPERVISED' is the domain's central trap -- productivity gains come from disciplined verification and lean context, not from removing checks or running agents unattended on production.
- ✓The Agent SDK builds custom internal agents for CI/CD checks and repetitive engineering tasks; headless/non-interactive mode lets those agents run without a human at the keyboard; custom slash commands and GitHub integration standardize common team tasks -- and all three should ENCODE the verify discipline, not bypass it.
- ✓Isolate a misbehaving system to its originating LAYER -- transport, integration/parsing, retrieval, or model output -- before changing anything; patching the wrong layer (prompt vs. integration code is the classic pair) fixes nothing and can mask the real defect.
- ✓Walk traces (request/response pairs, tool calls, retrieval hits, stop_reason, token usage) back to the EARLIEST deviation, not just the final symptom -- a fluent, well-formed answer can still trace back to a retrieval or integration failure several steps upstream.
- ✓Check operational basics first: truncated output means stop_reason: max_tokens (raise it -- not a security issue), a model version mismatch can silently change behavior, and context bloat late in a session degrades quality on its own.
- ✓A debugging pass isn't finished at the patch -- feed the confirmed fix back into evals and monitoring so the same issue is caught automatically next time, closing the loop with Domain 4/6 discipline.
- ✓A runbook maps symptom -> likely architecture cause -> first diagnostic action, built from real incidents, so a known issue's second occurrence is a five-minute lookup, not a from-scratch investigation; an escalation path (on-call -> platform team -> vendor support) removes guesswork about who to page for issues the runbook doesn't cover.
Check Your Understanding
Test what you learned in this lesson.
Q1.A team lead proposes removing the code-review step from their Claude Code workflow, arguing that since the agent already ran tests, human review is redundant overhead that's slowing the team down. How should this proposal be evaluated?
Q2.A support bot gives a fluent, confident, but factually wrong answer about a refund policy. Investigation shows the model's response was well-formed and the retrieval step returned chunks describing a different product's policy because an upstream query rewrite dropped a product filter. What is the correct classification and fix?
Q3.A Claude Code response is cut off mid-sentence in production. A developer suspects a prompt-injection attack and starts auditing the input for malicious content. What should an architect point out first?
Q4.A team wants to run a repetitive dependency-audit task across fifty repositories every night with no developer present, while preserving the same gather-context/plan/act/verify discipline used interactively. Which combination of mechanisms fits this need?
Q5.A recurring class of incident -- retrieval results going subtly off-topic after a scheduled re-index job -- has been independently diagnosed from scratch by three different engineers over six months, each spending several hours re-discovering the same root cause. What artifact would have prevented the repeated from-scratch investigations?
Practice This Lesson