Balancing Cost, Speed, and Quality When Optimizing
CoreOptimize workflows for efficiency and effectiveness · Difficulty 2/5
Explanation
The Three Levers, Together
Effective optimization balances cost, speed, and quality against the task's real requirements -- it does not maximize any single lever in isolation. A change that makes a workflow cheaper or faster is only a genuine optimization if quality stays at or above what the task actually needs. A change that ignores cost or speed entirely to chase marginal quality gains is not automatically correct either -- the goal is fit to the requirement, the same framing used elsewhere in the exam for model selection.
What Optimizing in Isolation Looks Like
- Cost-only optimization: routing everything to the cheapest model or skipping review to save spend, even on output where errors are costly.
- Speed-only optimization: skipping decomposition or review steps to finish faster, letting errors compound in a long task instead of being caught early.
- Both look like progress on their own metric while quietly degrading the dimension the task actually depends on.
Human Review Is a Governance Lever, Not Just a Speed Cost
Removing human review to save time on high-stakes output is not an optimization -- it is a governance and quality risk. The correct move is calibration (heavier review where stakes are high, lighter review where they are not), not uniform removal. This mirrors the broader principle: optimize the balance, not one corner of the triangle.
Common exam traps
- Optimizing only for cost or only for speed while quality drops below what the task needs -- the exam frames this as a failure of optimization, not a valid tradeoff.
- Treating "remove human review to go faster" as a legitimate efficiency gain; it is a governance/quality risk framed as speed, not a genuine optimization.
Key Takeaways
- Genuine optimization balances cost, speed, and quality against the task's real requirements -- not maximizing one lever alone
- Optimizing only for cost or only for speed while quality drops below what's needed is a common exam trap, not a best practice
- Removing human review on high-stakes output to save time is a governance/quality risk, not an optimization
- Calibrate review effort to stakes -- heavier where it counts, lighter where it doesn't -- rather than uniformly cutting it
Glossary Terms
The recurring architectural tension where an integration decision that improves accuracy (e.g., reranking, retrieving more chunks) typically adds latency and cost, and vice versa. The architect's job is not to eliminate the tradeoff but to make it explicit and justify the chosen configuration against whichever constraint the stated requirement names as dominant. Prompt caching a stable repeated context is the rare exception that improves cost and latency with no accuracy loss.
A design pattern that interrupts the agentic loop at defined checkpoints to request human review or approval before proceeding. Used for high-stakes decisions, irreversible actions, or cases where confidence is below threshold. Balances automation with oversight.
Related Concepts