Content Boundaries & Schema Design
CoreDesign content boundaries, schema output, session hygiene, and plugin management · Difficulty 2/5
Explanation
Content Boundaries
Across every interface, keep trusted instructions separate from untrusted data -- user input, retrieved documents, tool output. Untrusted content must be delimited clearly (for example, with XML-style tags) so that injected text inside the data cannot be read as an instruction to the model. This is the application-design half of a concern that also shows up as a security requirement: a prompt-injection payload hiding inside a retrieved document or tool result is only dangerous if the application fails to mark it as data rather than instruction.
Schema Design
When an application needs machine-readable output, define a JSON schema and use Structured Output or tool-forcing to get responses that conform to it. The design goal is a schema that is strict but not brittle:
- Strict enough that downstream code can parse the output reliably without defensive guesswork
- Not so brittle that a minor, harmless variation in phrasing or an edge case the schema didn't anticipate causes a hard failure
Getting this balance right is itself a design skill -- an overly rigid schema fails on inputs it should have handled; an overly loose one pushes parsing ambiguity back onto the caller.
Common exam traps
- Assuming any user-supplied or retrieved text is automatically safe to place directly in the prompt without delimiting it -- unsanitized untrusted content is a prompt-injection vector.
- Treating schema strictness as a pure good with no downside -- a schema so rigid it can't tolerate reasonable variation becomes brittle and fails on legitimate edge cases.
Key Takeaways
- Trusted instructions and untrusted data (user input, retrieved documents, tool output) must be kept separate and clearly delimited
- Unsanitized untrusted content inside a prompt is a prompt-injection vector
- Machine-readable output calls for a JSON schema plus structured output or tool-forcing
- A good schema is strict enough to parse reliably but not so brittle that it fails on reasonable variation
Glossary Terms
An attack where malicious content in external data (web pages, documents, user input) attempts to override the system prompt or hijack Claude's behavior. Mitigation: use XML tags to separate untrusted content from instructions, validate outputs, apply least-privilege tool access.
Guaranteed formatted output (typically JSON) from Claude. The most reliable method is to define a schema as a tool and set tool_choice to force its use — Claude's tool_use blocks are always valid JSON. Alternatively, use --output-format json with --json-schema in Claude Code CLI.
API parameter controlling how Claude selects tools. 'auto' (default): Claude decides whether to use tools. 'any': Claude must use at least one tool. 'none': Claude cannot use tools. '{type: tool, name: X}': Claude must use the specific named tool. Used to force structured output via a schema tool.
A Claude-native prompting pattern using XML-style tags (e.g., <document>, <instructions>, <example>) to clearly delimit sections of a prompt. Helps Claude unambiguously identify context, instructions, and data. Reduces prompt injection risk by separating instructions from untrusted input.
Related Concepts
Interfaces and Their Instruction Mechanisms
API/SDKs use the system parameter plus messages, tools, and params as their instruction mechanism
Session Hygiene & Plugin Management
Session hygiene means deliberately deciding what carries forward: compacting long threads, avoiding stale tool-output accumulation