2.3 Fact-Checking and Validation Techniques

Validation is the deliberate step of confirming an output before it's used. It does not happen automatically because an output reads well, and it is not the same activity as making the output look more polished. These two things get confused constantly: reformatting a paragraph, tightening its phrasing, or giving it a more formal tone changes its presentation, not its truth value. Treating a reformatting pass as if it were a verification pass is a category error — one step touches wording, the other touches facts, and doing the first does not accomplish the second.

This lesson builds directly on the previous one. Lesson 2.2 taught you to recognize where hallucinations, inconsistencies, and bias are likely to hide. Validation is what you actually do once you suspect one of those problems, or before you're willing to bet that none of them are present: you go find the authoritative source, the recomputed total, or the human reviewer that turns a suspicion into a confirmed fact. Recognizing risk and validating against it are two separate skills, and the exam tests both.

The right validation technique scales with the stakes and type of what's being confirmed. A citation needs a different check than a number, and a number needs a different check than a summary.

Notice that every row in the table below points to a concrete, external action — open a document, rerun an arithmetic check, hand it to a second person — not to an internal feeling about how trustworthy the output seems. That's the defining trait of genuine validation: it always terminates in something you did outside the text itself, never in a judgment about the text's tone or structure.

The exam returns to one scenario repeatedly because it cleanly tests the whole lesson: Claude confidently summarizes a new regulation and cites a specific subsection. What's the correct action before sending it to compliance? Verify that subsection against the official regulation text. Not send it because Claude sounded sure. Not reword it to sound more formal and send that instead. Confidence is not a verification substitute, and a more polished tone does not certify a citation's accuracy — the only thing that certifies the citation is opening the actual regulation and checking the actual subsection.

This scenario is a template, not a one-off. Swap "regulation subsection" for "court case," "internal policy number," or "a competitor's published figure," and the correct action is identical: open the real source and check it, every time a specific, checkable claim is about to be relied on by someone else.

A common shortcut is skipping verification because the output is "just internal" — a memo for a colleague, an analysis nobody outside the team will see. Stakes, not audience size, determine how much diligence a claim needs. An internal analysis that quietly informs a real decision — a budget call, a staffing plan, a go/no-go — carries real consequences even though only a handful of people will ever read it. "Internal" describes distribution, not accuracy requirements.

• •Reformatting or making an answer sound more formal is never a substitute for verification — it changes wording, not truth.
• •"It's just internal" does not exempt a claim from checking if a real decision depends on it.
• •The validation technique should match the claim type: source text for citations, recomputation for numbers, source-fidelity check for summaries, human review for anything customer-, legal-, or compliance-facing.

A useful gut-check when you feel the pull of the "it's just internal" shortcut: ask what happens if the claim turns out to be wrong. If a wrong number in a supposedly low-stakes internal memo would still cause someone to make a bad staffing decision or approve the wrong budget line, the fact that the memo never left the building was never the relevant variable. Audience size measures how many people would see the error, not how much the error would cost.

Two more techniques round out the validation toolkit from this lesson, and both work by making Claude's reasoning -- not just its conclusion -- visible for you to check.

Quote first, then analyze. For a long document, ask Claude to pull the exact supporting quotes before drawing any conclusion. A conclusion tied to a named quote can be checked in seconds against the source; a conclusion with no attached quote asks you to take the analysis on faith.

The difference shows up clearly on a real task. Ask Claude to flag any clause in a 30-page vendor contract that lets the vendor unilaterally raise prices. An ungrounded answer might read: "Section 7 allows the vendor to increase pricing at their discretion with limited notice, creating meaningful cost-exposure risk." That sounds like a real finding, but there's nothing underneath it to check -- is there really a Section 7, does it really say that, what does "limited notice" mean in days? You'd have to reopen the whole contract to find out, which defeats the point of asking Claude to find it for you.

A grounded answer instead reads: "Section 7.2 states: 'Vendor may adjust the Fees upon thirty (30) days' written notice to Customer, at Vendor's sole discretion.' This is a unilateral price-increase clause -- it requires only notice, not customer consent, and the window is 30 days." Now the claim is falsifiable in ten seconds: open the contract to 7.2, confirm the quoted sentence exists verbatim, and confirm the interpretation matches it. Same underlying clause, same conclusion -- but only one version can be checked without redoing the work yourself.

Best-of-N comparison. Send the same request through several independent runs and line up what comes back. Where those runs agree, that agreement is a genuine confidence signal. Where they diverge, you've found a soft spot the model doesn't reliably know the answer to -- exactly the kind of claim that needs a human look before anyone relies on it. Ask that same contract question in three fresh conversations: if all three flag Section 7.2 with the same 30-day figure, the convergence means something. If one run says 30 days, another says 60, and a third skips the notice period entirely, you've just learned exactly which detail to verify by hand -- something a single confident-sounding run would never have told you.