LLM Risk & Failure Mode Taxonomy

An architect is expected to name the failure modes of LLM systems and design around each of them, rather than treating "the model might be wrong" as a single undifferentiated risk.

The Named Failure Modes

• Hallucination -- confident but fabricated output; mitigate with retrieval grounding, citations, constrained claims, and "say I don't know" allowances.
• Non-determinism -- the same input can yield different output; design for it (evals, human review on high stakes) rather than assuming reproducibility.
• **Prompt Injection / jailbreak** -- malicious instructions in untrusted content hijacking model behavior (see Domain 5b).
• Data leakage -- the model surfacing PII or secrets it shouldn't; scope data access and validate output.
• Model mismatch / drift -- behavior changes across model versions; pin versions and re-run evals.
• Over-reliance / automation bias -- humans trusting confident output uncritically; counter with transparency and human validation.

Why the Taxonomy Matters

Each failure mode has a distinct mitigation. Grounding fixes hallucination but does nothing for data leakage; scoping data access fixes leakage but does nothing for drift; pinning versions fixes drift but does nothing for automation bias. A scenario item that asks "what's the correct mitigation" is testing whether the right failure mode has been identified in the first place, not just whether a generically "safe-sounding" answer was chosen.

Common exam traps

• Treating a fluent, confidently-worded answer as evidence of correctness. Confidence is not correctness -- high-stakes claims must be grounded and verified regardless of how the model states them.