Validation Loop (Retry-with-Feedback)
PromptingDefinition
A pattern where structured output that fails validation is sent back to Claude with the specific error, requesting a correction. More effective than silent retries because Claude uses the error feedback to understand and fix the problem. Typically capped at 2-3 retries.
Example Usage
If JSON.parse() throws, send the error message back: 'Your output failed validation: {error}. Please correct and return valid JSON.'
Why It Matters for the CCA-F Exam
Validation loops are core to Domain 4 agentic reliability questions. The exam distinguishes between silent retries (resample without feedback — lower quality) and retry-with-error-feedback (the correct pattern). Expect questions about when to use structured outputs vs. validation loops, and how to bound the loop to avoid infinite retries.
In Depth
A validation loop — also called retry-with-feedback or retry-with-error — is an agentic pattern where structured output that fails validation is returned to Claude along with the specific error message, requesting a correction. The key insight is that *why* the output failed is more actionable than the fact that it failed: instead of silently re-sampling, the loop feeds the error back as context, giving the model the information it needs to fix the specific problem.
The pattern addresses a fundamental tension in production systems: language models are highly capable at generation but not guaranteed to produce syntactically or semantically valid output on every call. Structured output modes (see Structured Output) enforce schema conformance at the API level and should be the first line of defense, but they do not validate semantic constraints — required fields having plausible values, enumerations containing only legal values, cross-field consistency, or business-rule compliance.
A minimal validation loop works in three steps. First, call Claude and receive output. Second, validate the output against your schema and business rules. Third, if validation fails, construct a follow-up message that includes the original output, the validation error(s), and a clear request to correct and regenerate — then repeat from step one. Industry practice caps retries at 2–3 iterations; beyond that, escalate to a human or return a structured error rather than looping indefinitely.
Critical implementation details: (1) Include the full error message in the retry prompt — "Your output failed JSON parsing: 'Expecting value: line 1 column 42'" is far more useful than "Invalid output, try again." (2) Echo back the original output so Claude can see exactly what it produced and where it went wrong. (3) Maintain the conversation history so the model has full context. (4) Apply a backoff and cap to avoid infinite loops and runaway token costs.
The pattern pairs naturally with Self-Critique — a self-critique pass before the first validation attempt can catch obvious issues before the external validator ever runs — and with Evaluator-Optimizer Pattern, which formalizes a generator/critic architecture at a higher level of abstraction.
How It Compares
N/A
Example
Retry-with-error-feedback: the error is passed back as a new user message; loop is capped at MAX_RETRIES.
import anthropic, json
client = anthropic.Anthropic()
MAX_RETRIES = 3
def call_with_validation(messages, schema_validator):
for attempt in range(MAX_RETRIES):
response = client.messages.create(
model="claude-opus-4-8",
max_tokens=1024,
messages=messages
)
content = response.content[0].text
try:
data = json.loads(content)
schema_validator(data) # raises ValueError on semantic failure
return data
except (json.JSONDecodeError, ValueError) as e:
if attempt == MAX_RETRIES - 1:
raise RuntimeError(f"Validation failed after {MAX_RETRIES} attempts") from e
# Retry with the error as feedback
messages = messages + [
{"role": "assistant", "content": content},
{
"role": "user",
"content": (
f"Your output failed validation with this error:\n{e}\n\n"
"Please correct the problem and return valid output."
)
}
]
return NoneHow It's Tested & Common Confusions
N/A
Frequently Asked Questions
Why is retry-with-feedback better than silent re-sampling?
Silent re-sampling asks Claude to regenerate output without any information about what went wrong. The model is likely to produce a similar incorrect output. Retry-with-feedback includes the specific error, which gives the model precisely the information it needs to fix the problem — typically producing a correct result within one or two additional attempts.
Should I use structured output modes instead of a validation loop?
Use both in layers. The `output_config: {format: {type: "json_schema", schema: ...}}` structured output API enforces syntactic schema conformance at the model level and should be your first defense. A validation loop on top handles semantic constraints — business rules, cross-field consistency, value plausibility — that a JSON schema cannot express.
How many retries are appropriate before escalating?
Industry practice caps validation retries at 2–3 attempts. Beyond that, the same error recurring likely indicates a structural problem — an ambiguous instruction, a schema the model cannot satisfy, or an input that cannot be processed — rather than a transient generation issue. Escalate to a human review queue or return a structured error response rather than looping indefinitely.