1.3 End-to-End Architecture & Feedback Loops

Every pattern in Lesson 1.2 — augmented LLM, workflow, agent — describes what happens at the CENTER of a system: the reasoning step. But a production system is never just that center. Ask yourself: where does the data that reaches the model come from, and has anyone checked it's trustworthy? What happens to what the model produces before anyone downstream relies on it? And how would you ever find out if quality quietly got worse after a model upgrade last Tuesday? None of those questions are answered by picking a pattern. They're answered by designing the shape AROUND the pattern.

That shape has a name, and it's the second thing Task Statement 1.3 wants you fluent in: input → processing → output → feedback loops. Processing is the pattern you already chose in Lesson 1.2. The other three stages are what turn that pattern into an actual system that survives contact with the real world, and each one has its own concerns that the exam expects you to name specifically, not just gesture at.

The input stage covers everything that happens before the model ever reasons: ingestion (getting the raw data in), validation (checking it's well-formed and expected), and retrieval of grounding context if the design needs it (fetching the documents or records the model will reason over — a Domain 3 topic in depth, but the decision that grounding is needed at all is made right here at input design).

But the concern the exam leans on hardest at this stage is a security-adjacent one that belongs squarely in architecture thinking: separating TRUSTED instructions from UNTRUSTED data. Your system prompt, your developer instructions — those are trusted; they tell the model what to do. A customer's message, a scraped web page, the contents of a retrieved document — those are untrusted DATA the model should reason ABOUT, never instructions it should obey. If your architecture doesn't draw that line clearly, you've built a system where anyone who can influence the untrusted data — a customer typing a message, an attacker planting text in a document you'll retrieve — can potentially inject instructions the model treats as if you'd written them yourself.

This is exactly the kind of thing that's easy to nod along with in the abstract and then blur in an actual design, e.g. concatenating retrieved document text directly into the same prompt block as your instructions with no structural separation. A well-designed input stage keeps that boundary explicit and structural, not just a matter of hoping the model figures out which part was which.

The processing stage is a short entry in this lesson precisely because you already spent all of Lesson 1.2 on it: it's the chosen pattern (augmented LLM, workflow, or agent), the model selection for that pattern, and the prompt and context assembly that feeds it. Nothing new to add here except a reminder — everything you already know about picking the least-complex sufficient pattern is exactly the content of this stage.

The output stage is where a design earns or loses trust with everything downstream of it. A raw model response is text; a production system almost never wants raw text, it wants a STRUCTURED OUTPUT CONTRACT — a defined shape (JSON schema, a specific format) that downstream code can rely on without having to guess. Alongside the contract sits validation (does this response actually satisfy the contract?) and defensive parsing (what does your code do when it DOESN'T — because a probabilistic model will eventually produce something that doesn't match, and "the code crashes" is not an acceptable answer). Only once a response is validated does it get delivered to whatever downstream system is waiting on it.

This is the stage most likely to be missing from a design that otherwise looks complete, and it's the one Task Statement 1.3 treats as non-negotiable. Feedback loops are evaluation, monitoring, and observability — the mechanisms that feed measured quality back into iteration. Concretely: how do you know, with evidence rather than a hunch, that the system is still performing at the bar it was designed to hit?

Here's the argument for why this can't be optional, and it's worth internalizing rather than memorizing: Claude's outputs are non-deterministic. The same input can produce a different response on different runs, a model version upgrade can shift behavior in ways that aren't obvious until you measure them, and the data flowing through retrieval can drift as the underlying knowledge base changes over time. None of those are hypothetical edge cases — they are the normal operating conditions of any system built on a probabilistic model. Without a feedback loop, you cannot detect a regression when a new model version subtly changes behavior, you cannot catch retrieval drift when your knowledge base quietly goes stale, and you cannot PROVE — to yourself, to a stakeholder, to an auditor — that the system still meets the SLA you promised in Lesson 1.1's requirements-gathering step.

This is what separates an architecture from a demo. A demo proves the approach can work once, in front of you, on the examples you happened to try. An architecture proves — continuously, with evidence — that it keeps working, on inputs you didn't hand-pick, after changes you didn't personally review.

There is one input-stage mistake that shows up more often than any other in real Claude reference architectures, and it deserves a name of its own precisely because it is so easy to ship without noticing: building a retrieval index over data that is actually LIVE, MUTABLE STATE, and then querying that index as if it were a stable knowledge base. The architecture looks correct. The retrieval pipeline runs without errors. The demo answers every question you try. The failure only shows up once two retrieved chunks disagree with each other, because the real-world fact each chunk describes changed between the moment it was indexed and the moment it was retrieved.

The root cause is a reflex, not a knowledge gap: once a team has RAG working for one thing (product documentation, policy text, historical records), it becomes the default answer to "the model needs information it doesn't already have" — even for information whose truth value can change in the next sixty seconds. Documentation is stable enough that a nightly-refreshed index is a reasonable tradeoff. A record of CURRENT state — an order's status, a room's availability, a ticket's owner — has no refresh interval short enough to be safe, because the ground truth can move between the refresh and the query that reads the stale copy.

Take a delivery-tracking assistant as the worked example. A team indexes each shipment's scan events into a vector store overnight and has the assistant retrieve the most relevant chunks to answer a customer's "where is my package" question. At 8:00am the index still shows the package's last known scan as "out for delivery," captured in yesterday's snapshot. At 9:40am the courier actually delivers the package and scans it as "delivered" — an event that happens entirely inside the live courier system and has no path into the vector store until the next nightly refresh. At 10:15am the customer asks the assistant for an update, and the assistant retrieves the stale "out for delivery" chunk with complete confidence, because from the retrieval pipeline's point of view nothing is wrong — the chunk matches the query, the embedding similarity is high, the citation looks legitimate. The customer is told their package is still coming. It was sitting on their porch for ninety minutes already.

The fix is architectural, not a tuning knob. A shorter refresh interval, a better embedding model, or an added reranking step do not touch the actual defect, because the defect is that the wrong storage mechanism is being used for this specific class of data in the first place. The correct fix is to QUERY THE LIVE SYSTEM OF RECORD DIRECTLY — a database read, or a call to the courier's live tracking API, or an internal service call — for any fact whose truth value can change faster than your refresh cycle, and to reserve retrieval for content that is genuinely stable across that same window (the carrier's standard delivery-window policy, return instructions, packaging FAQs). In the worked example, that means the "where is my package right now" step becomes a live API call to the courier's tracking system, made at request time, every time — while a question about the general delivery policy still correctly goes through retrieval, because policy text doesn't change between the moment it's indexed and the moment a customer asks about it.

The most useful skill from this lesson is a habit, not a fact: when you're handed a design — yours or someone else's — run it through all four stages and ask what's missing. Most flawed designs on the exam aren't wrong about the PATTERN (the processing stage); they're incomplete about the other three.

• 1.INPUT — Is there validation? Is grounding context retrieved if needed? Is untrusted data structurally separated from trusted instructions?
• 2.PROCESSING — Is the pattern from Lesson 1.2 the least complex one that satisfies the Lesson 1.1 requirements?
• 3.OUTPUT — Is there a structured-output contract? Is the response validated against it? Is there defensive parsing for when it isn't?
• 4.FEEDBACK — Is there evaluation, monitoring, and observability that would catch a regression, drift, or an SLA breach with evidence, not assumption?

A scenario that describes a system with a great processing pattern but no mention of validation, contracts, or evaluation is not a trick question about the pattern — it's testing whether you notice the other three stages are simply absent. That's the recurring shape of Task Statement 1.3 items.