The SDLC for Claude Applications: Evaluation, Pinning & Monitoring

The Standard Life Cycle, Still Applies

Claude applications live inside a normal systems/software development life cycle: requirements -> design -> implementation -> testing/evaluation -> deployment -> operation -> maintenance. Nothing about building on an LLM exempts a project from this frame.

The LLM-Specific Wrinkle

What changes is that model behavior is non-deterministic and model-version-dependent. A single equality assertion ("output == expected string") is the wrong tool for testing a stochastic system. The life cycle has to add:

• Evaluation as a first-class phase -- build evals with success criteria (a rubric, a scoring model, a tolerance band) instead of exact-match tests. This is where Domain 4 (evaluation, testing, optimization) plugs into the life cycle.
• Model-version pinning and regression testing -- a new model release can change behavior even without an API change, so re-run evals before upgrading rather than floating to the newest version.
• Monitoring in production -- track quality, latency, token cost, and error rates over time, since a system that passed evaluation at launch can still drift or degrade once it's live.

Common exam traps

• Treating testing/evaluation as a single pre-launch gate rather than a recurring activity that also gates every subsequent model or prompt change.
• Assuming a passing eval at launch means monitoring is optional afterward -- production behavior (quality, latency, cost, errors) still needs continuous observation.
• Forgetting that model-version changes are a life-cycle event requiring the same regression discipline as a code change, even when no code was touched.