The SproutRoute model evaluated in this benchmark fell below the 8.0 safety floor.
Evaluation infrastructure / release operations
AI Eval Control Tower
A cloneable model-evaluation system that turns quality, safety, cost, latency, and score provenance into a PM-readable launch decision.
Artifact replayBrowser-local result import
- Models
- 7 registered
- Domains
- 3 workflows
- Finding
- Safety floor fail
- Use
- Release gate
- Decision
- Combine model quality, safety, latency, cost, and provenance in one release verdict.
- Why
- Separate notebooks and spreadsheets left launch owners without a comparable evidence chain.
- Result
- The benchmark exposed a production-model safety-floor failure and identified a stronger routing option.
Product decisionOperating mechanism
Make model selection a repeatable product decision.
This case shows how I turn AI quality from a subjective demo review into an operating mechanism: eval rubrics, safety floors, latency/cost tradeoffs, score provenance, and explicit launch decisions. The Control Tower serves as the enterprise platform's decision plane.
See its role in the platform operating modelQuick readOperating layer
Most AI teams still lack a clear operating layer for model decisions.
One spreadsheet for prompts, one notebook for model experiments, another chart for latency, and a separate launch debate creates fragmented evidence. This tool makes the decision chain inspectable.
Sonnet 4.6 led the field, but with latency and cost tradeoffs.
The matrix makes "good enough and cheaper" a provable product decision.
Secret-free PR checks stay safe; live OpenRouter runs happen only by manual workflow.
BenchmarksDecision
Seven models, one rubric, one decision.
Quality leader
Sonnet 4.6
Quality 87.8, safety 9.04, high latency and cost.
Best tradeoff
Gemini 2.5 Flash
Cleared safety, faster than production, lower cost.
Blocked path
Haiku 4.5
Evaluated model scored below the safety floor.
Evidence
Judge citations
Car seat law, venue closure, and infeasible routing errors were surfaced.
ArtifactLive build
The evidence explorer is part of the case study.
The explorer derives candidates, dimensions, verdicts, latency, cost, safety floors, judge metadata, and recommendations from selected JSON artifacts. A validated local import supports new evidence without transmitting the file.
Open evaluation evidence Inspect the runtime trace layerTest reportsArtifact replay
Compare the results behind a release decision.
Select a recorded evaluation to compare quality, safety, latency, and cost across models. Open your own results locally to review another run against the same release criteria.