The first committed span records the healthy retrieval step.
Implemented platform capability / runtime evidence
Find the broken step, not just the bad score.
The Eval Control Tower accepts span-level traces from evaluation runs and agent workflows, rendering sequence, status, relative offsets, fallback, cost, and privacy-safe detail in one operator workflow.
Artifact replay + local importCommitted forced-failure and load artifacts
- Contract tests
- 4 passing
- Burst exercised
- 150k spans
- Errors retained
- 150
- Raw payload fields
- 0
- Decision
- Use a lightweight JSON span contract for validation; keep OpenTelemetry as the scaled transport, not a prototype dependency.
- Why
- Aggregate quality scores cannot isolate the retry, fallback, model call, or agent edge that consumed the run.
- Evidence boundary
- The default view reads committed artifacts. Relative offsets are disclosed, imports stay browser-local, and the load report is an in-process collector simulation.
Operator workflowFailure first
One trace answers four operating questions.
Sequence
What ran?
Parent relationships and offsets reconstruct the evaluation or agent execution path.
Status
What broke?
Errors, fallbacks, and skipped spans remain visible and filterable.
Latency
What blocked?
The waterfall makes the critical path and parallel branches legible.
Cost
What spent?
Token pricing attributes eval cost to the exact model step that incurred it.
Incident sampleForced failure
The default trace starts with something wrong.
The next span retains POLICY_SERVICE_UNAVAILABLE without raw request content.
The final span records policy judge unavailable as the fallback reason.
The failures filter narrows the committed trace to the error span.
Scale contractDegrade safely
Observability must not become an outage dependency.
The 150,000-span burst simulation fills a 60,000-span buffer, retains all 150 error spans, and drops only low-priority successes. Tenant validation occurs before trace access; detail fields use an allowlist. The production build and desktop/mobile operator paths were verified, while a real orchestrator-run capture remains the next evidence gate.
Open the Control Tower caseInspect the trace sourceEvidence contractArtifact replay
Trace claims resolve to exact JSON.
Run npm test && npm run trace:failure && npm run trace:load. The explorer displays the source commit, run ID, timestamp, artifact hash, reconstructed-offset disclosure, load limitations, and validated local import boundary.