2.1 Evaluating Accuracy and Completeness

Claude produces fluent, confident text. That single fact is the reason this entire domain exists, and it's worth sitting with before anything else: the model's writing quality tells you nothing about whether its content is true. A hallucinated statistic reads exactly as smoothly as a verified one. A dropped requirement can be buried inside three paragraphs of otherwise well-organized prose. Before an output is used for anything that matters, it has to be judged against what the task actually required — not against a vague, in-the-moment impression of whether it reads well.

This is the defining mindset of the Associate role, and it's worth naming directly: diligence. The person using Claude stays accountable for the output. Confidence in Claude's wording is never itself evidence that the content is correct. Everything in this domain — checking accuracy, catching hallucinations, fact-checking, deciding when a human needs to sign off, editing, and choosing the right format — is diligence applied at a different stage of the same workflow.

Anthropic's guidance on defining success criteria applies directly to evaluation: decide up front what "good" looks like, then check the output against that definition. A vague standard like "does this look okay?" can't be verified consistently — two reviewers might disagree, or the same reviewer might miss something on a second read. An explicit, checkable criterion removes the ambiguity. "Must cite the correct regulation subsection." "Must stay under 200 words." "Must not recommend a refund." Each of these can be answered with a yes or no.

The sequencing matters as much as the content of the criteria. Success criteria have to be set before the draft is read, not after. If you wait until an appealing, well-written draft is already in front of you, the draft quietly shapes the standard instead of the standard shaping your judgment of the draft — you'll rationalize why the parts it got right are the parts that mattered.

Once success criteria are set, a practical five-part checklist turns them into a repeatable review. Run every output-in-question through all five dimensions — not just the one that happens to catch your eye first.

Notice that accuracy is only one-fifth of the checklist. An output can get every individual fact right and still fail the review — by skipping part of a multi-part ask, by answering a slightly different question than the one posed, by contradicting itself between paragraphs, or by being pitched at the wrong reader. The checklist exists precisely because these are independent failure modes, not one problem wearing different masks.

Completeness is the checklist item most often faked by good writing. A response can read as thorough — well-organized, confident, detailed — while quietly skipping one part of a multi-part request. Checking completeness means going back to the original request and confirming each requested part is actually present in the output, not just that the response *feels* comprehensive on a read-through. If someone asked for a summary, a risk assessment, and a recommendation, and the output delivers a beautifully written summary and recommendation but no risk assessment, it is incomplete — no matter how polished the two delivered pieces are.

Relevance has the same trap in a different shape. Claude can produce a fluent, on-topic-sounding answer to a slightly different question than the one that was actually asked. Checking relevance means re-reading the actual question and confirming the output answers *that* question, not a plausible neighbor of it — a common failure when a prompt contains two related sub-questions and the response thoroughly answers only the easier one.

• •Reread the original request line by line and tick off each discrete part — don't rely on a general impression of thoroughness.
• •Restate the actual question in one sentence and check the output answers that sentence, not an adjacent one.
• •Treat a well-organized, confident answer as a reason to look closer, not a reason to skip the check.

Everything covered so far in this lesson -- setting success criteria before reading a draft, running the five-part checklist, catching completeness dressed up as thoroughness, catching relevance dressed up as a nearby answer -- is a single named competency in Anthropic's AI Fluency framework: Discernment, the discipline of judging output against what was actually requested, what the sources actually say, and what the field considers acceptable. The success criteria and the checklist aren't separate tricks bolted onto this lesson; they're Discernment made concrete and checkable.

This course names Diligence explicitly elsewhere -- iteration in Lesson 1.3, ethical use in Lesson 6.4 -- as the competency of staying responsible for an outcome and using AI effectively, ethically, and safely. Discernment is Diligence's quieter partner, and the exam expects you to tell them apart cleanly rather than treat both as "being careful with Claude."

The two competencies compound rather than operate independently, and the exam's trickiest scenarios exploit the gap between having one without the other. Discernment without Diligence looks like an Associate who knows the five-part checklist cold but, rushed before a deadline, skips running it and ships the draft anyway -- the skill existed, the discipline to apply it under pressure didn't. Diligence without Discernment looks like an Associate who dutifully reviews every output every time, but only checks tone and length, missing a fabricated citation, because they never learned what the checklist actually is -- the discipline existed, the skill to execute it well didn't. Both produce the same visible failure, an error reaching production, but the fix differs: the first needs a workflow gate that forces review regardless of time pressure; the second needs training on what to actually check.

• •A scenario turning on "they didn't realize completeness and relevance are separate failure modes" is testing Discernment -- a skill gap.
• •A scenario turning on "they knew better but skipped the check because the deadline was tight" is testing Diligence -- a discipline gap.
• •Recognizing which one a scenario is testing is itself a form of Discernment, applied one level up to your own review process.

Task Statement 2.1 questions tend to describe an output that reads impressively — formal tone, confident phrasing, good structure — and then ask what you should do before relying on it. The wrong answers lean on that polish as if it were proof. The right answer always routes back to the same two moves: had success criteria been set, and does the output actually satisfy all five checklist dimensions, especially completeness and relevance, which are the two most often faked by fluent writing.

Hold onto this habit for the rest of the domain: every later skill — spotting hallucinations, fact-checking, deciding on human review, editing, and choosing a format — assumes you've already done this first pass. Evaluation against explicit criteria is the gate everything else in Domain 2 walks through.