2.2 The Systems Life Cycle for Claude Applications
2.2.1 The Standard SDLC, Still Applies
Claude applications don't get a pass on ordinary software process. They live inside the same systems/software development life cycle as any other application: requirements, design, implementation, testing/evaluation, deployment, operation, and maintenance. Nothing about building on top of an LLM exempts a project from this frame -- what changes is what has to be added inside it.
- 1.Requirements -- the business, functional, and infrastructure requirements from Task Statement 2.1.
- 2.Design -- pattern, model tier, realtime/batch choice, integration surface.
- 3.Implementation -- the actual Messages API integration, tool definitions, error handling.
- 4.Testing / evaluation -- a first-class phase, not a formality (see 2.2.3).
- 5.Deployment, operation, maintenance -- shipping it, running it, and keeping it correct over time.
2.2.2 The LLM-Specific Wrinkle: Non-Determinism
The reason this life cycle needs anything added at all is that Claude's behavior is non-deterministic and model-version-dependent. You cannot unit-test a stochastic output with a single equality assertion the way you'd test a pure function -- "assert output == expected_string" is the wrong tool when the same prompt can legitimately produce several acceptable phrasings of a correct answer.
2.2.2 -- Key Concept
Non-deterministic, model-version-dependent behavior is the reason the SDLC needs LLM-specific additions: evaluation as a first-class phase, model-version pinning with regression testing, and continuous production monitoring. None of the three is optional bolt-on work.
2.2.3 Evaluation as a First-Class Phase
Because a single expected string can't grade a stochastic output, evaluation has to be built with success criteria -- a rubric, a scoring model, a tolerance band -- instead of exact-match tests. This is where the evaluation domain plugs directly into the life cycle: it isn't a side activity, it's the testing phase, restated for a system that answers in natural language.
# A minimal eval loop: success criteria, not exact-match asserts
results = []
for example in held_out_set:
output = client.messages.create(
model="claude-sonnet-4-5", max_tokens=512,
messages=[{"role": "user", "content": example.prompt}],
)
score = grade(output.content[0].text, rubric=example.rubric) # not output == expected
results.append(score)
pass_rate = sum(results) / len(results)
assert pass_rate >= 0.90, "Regression against the defined success bar"2.2.4 Model-Version Pinning & Regression Testing
A new model release can change behavior even when your code hasn't changed at all. The life-cycle discipline this demands is pinning an explicit model version and re-running your evaluation suite before adopting a new release -- treating a model upgrade as a life-cycle event with the same regression discipline as a code change, not a free, automatic improvement.
| Approach | Behavior |
|---|---|
| Floating alias (e.g. "latest") | Can silently change underneath you on Anthropic's release schedule |
| Pinned model ID + gated upgrade | Behavior stays reproducible; you upgrade only after re-running evals and reviewing the diff |
The life-cycle answer to model-version risk: pin, then upgrade deliberately.
2.2.5 Production Monitoring Is Not Optional
Passing your evaluation suite at launch tells you the system worked at that moment, on that model version, against that test set. It says nothing about whether quality is still holding a month later, or under real user input the eval set didn't anticipate. Monitoring in production is the ongoing half of the testing/evaluation phase -- it doesn't end at deployment.
- •Quality -- is output still meeting the success criteria the eval suite checked at launch?
- •Latency -- are response times still within the requirement's stated SLA?
- •Token cost -- is per-request spend tracking the budget the requirement set?
- •Error rate -- are 429s, 5xxs, or malformed outputs trending up?
Exam trap
Treating testing/evaluation as a single pre-launch gate is a common trap. A system that passed evaluation at launch can still drift or degrade once live -- monitoring quality, latency, cost, and error rate is a recurring life-cycle activity, not a box checked once.
2.2.6 Life Cycle Events, Not Just Launch Events
A useful reframe for this whole task statement: every one of its additions -- evaluation, version pinning, monitoring -- exists because a Claude application has more "events" that can change its behavior than a traditional deterministic service does. A traditional service's behavior changes when someone deploys new code. A Claude application's behavior can change when someone deploys new code, when someone edits a prompt, when Anthropic ships a new model version, or when a connected plugin updates -- and only the first of those four shows up in a typical deployment log unless you deliberately instrument for the other three.
| Event | Traditional service | Claude application |
|---|---|---|
| Code deploy | Triggers behavior change | Triggers behavior change |
| Prompt edit | N/A | Triggers behavior change -- needs the same review/versioning as code |
| Model version update | N/A | Can silently change behavior -- needs pinning + regression testing |
| Plugin/MCP-server update | Occasionally relevant (library upgrade) | Can silently change behavior -- needs tracking + evals |
A Claude application has more behavior-changing events than a traditional deterministic service; the life-cycle additions in this task statement exist to catch all of them, not just code deploys.
Holding this reframe in mind is often enough to answer an exam question correctly even without recalling the specific term: if a scenario describes any of these four events happening without an accompanying eval/monitoring/review step, the gap is the life-cycle addition that event is missing.
Key Takeaways
- ✓Claude applications follow the standard SDLC: requirements, design, implementation, testing/evaluation, deployment, operation, maintenance.
- ✓Non-deterministic, model-version-dependent behavior means a single equality assertion cannot test a stochastic output.
- ✓Evaluation is a first-class phase built on success criteria (rubrics, scoring models, tolerance bands), not exact-match tests.
- ✓Model-version pinning plus regression testing gates every upgrade, not just the initial launch -- a new release is a life-cycle event.
- ✓Production monitoring of quality, latency, token cost, and error rate is an ongoing activity, not a one-time launch check.
Check Your Understanding
Test what you learned in this lesson.
Q1.Why can't a Claude feature be tested with a single "output == expected_string" assertion?
Q2.A team's evals passed cleanly at launch three months ago and haven't been looked at since. What life-cycle gap does this describe?
Q3.Anthropic ships a new default model version. What should happen before an application adopts it in production?
Q4.Which SDLC phases does a Claude application still go through, per the exam blueprint?
Practice This Lesson