The fixed synthetic SproutRoute workload clears the 30% cost target without relying on cache hits.
Implemented platform capability / model access and economics
Govern the model boundary once.
An OpenAI-compatible gateway moves provider choice, data protection, semantic reuse, fallback, quota, and cost attribution out of product code. SproutRoute has a tested, reversible integration path.
Local executionCommitted deterministic benchmark
- Gateway tests
- 47 passing
- Cold cost delta
- −66.41%
- Warm cost delta
- −82.81%
- Quality delta
- −1 point
- Decision
- Centralize routing policy while leaving product behavior and a direct-provider rollback with the application team.
- Why
- Duplicated provider logic obscures spend, weakens data controls, and makes fallback behavior impossible to govern consistently.
- Evidence boundary
- The public workbench executes policy locally and stops at a deterministic provider stub. Cost and latency figures come only from the committed synthetic benchmark.
Request pathFail closed
Policy sits between identity and inference.
Attribute
Authenticate
Hashed service credentials scope tenant, service, environment, role, model, and data class.
Protect
Classify
DLP blocks restricted data, redacts approved identifiers, and enforces regional routing before provider access.
Decide
Route
Versioned policy selects a model, samples quality, opens circuit breakers, and advances a bounded fallback chain.
Conserve
Reuse
Tenant-safe semantic caching and request, token, concurrency, and spend limits protect compute margin.
Measured evidence50-request workload
The report separates proof from projection.
A controlled 50% reuse pattern shows the economic upside of tenant-safe semantic caching.
The in-process load harness holds the shared envelope, denies 408 excess requests, and returns no cross-tenant cache result.
LLM_GATEWAY_URL and a scoped credential opt SproutRoute in; leaving either unset preserves the direct route.
Next decisionExternal validation
Do not promote on simulated economics alone.
The next gate is a credentialed three-provider replay through LiteLLM and Redis, including one real provider failure, measured network p95, and a quality review using the same frozen workload. The implementation is ready for that decision without claiming it has already happened.
Next capability: runtime tracesReturn to the operating modelEvidence contractLocal execution
Reproduce the claims from the source repository.
Run npm test && npm run benchmark && npm run load. The public workbench exposes the exact source commit, artifact hashes, supported claims, and limitations; it never presents the deterministic provider stub as a model answer.