System vs. User Placement & Prompt Templates
CoreDesign system prompts, templates, and guardrails · Difficulty 2/5
Explanation
System vs. User Placement
An architect separates prompt content by stability, not just by role:
- **System Prompt**: stable rules, role, tone, and constraints that hold across every request
- User message: the specific request and per-request data that varies call to call
This separation does double duty. It keeps behavior consistent across requests, and it keeps the stable content together as a single block -- which matters for Prompt Caching, since only a stable, unchanging prefix is cacheable.
Templates
Templates parameterize the variable parts of a prompt (the specific document, user query, or record being processed) while keeping the surrounding scaffold -- instructions, format spec, examples -- constant. This improves two things at once:
- Consistency: the same scaffold produces comparable output across many requests
- Cache hit rates: a constant scaffold is exactly the stable prefix that caching rewards
Design Pattern
Treat the System Prompt as a reusable, versioned artifact -- not an ad-hoc string assembled per request. Changes to it should go through the same review/eval discipline as code.
Common exam traps
- Mixing stable and per-request content together in one blob, which both hurts consistency and breaks the cacheable prefix.
- Treating templates as a UX convenience only, missing that they also drive cache economics.
Key Takeaways
- Stable rules, role, tone, and constraints belong in the system prompt; the specific request and data belong in the user message
- This separation improves consistency and keeps the stable content cacheable
- Templates parameterize variable parts while holding the scaffold constant
- Treat system prompts as versioned, reviewed artifacts, not ad-hoc strings
Glossary Terms
The initial instruction set provided to Claude that defines its behavior, role, constraints, and operational context for an entire conversation. Set via the 'system' parameter in the API. Processed before the user turn and shapes all subsequent responses.
The practice of defining Claude's identity and expertise in the system prompt to anchor its behavior. Roles like 'You are an expert security auditor' improve response quality for domain-specific tasks by activating relevant knowledge and behavioral patterns.
A Claude API feature that caches frequently-used prompt content (system prompts, large documents, tool definitions) to reduce cost and latency on repeated API calls. Cached tokens are billed at a discounted rate. Cache has a TTL that resets on each use. Must be enabled by marking content with cache_control.
Related Concepts
Prompt Guardrails vs. Deterministic Controls
Prompt-level guardrails (refusals, content policy, escalation) shape behavior probabilistically, not deterministically
Prompt Caching & Stable-Prefix Ordering
A stable prefix (system prompt + policy + few-shot block) can be cached: reads are cheap, writes carry a slight premium