All work

AI infrastructure / Case study

Synapse Router

Escalate to a stronger model when confidence drops.

Open the live demo
Type
Open-source engineering
Release
v1.1.0
Stack
Python 3.12 · Multi-Model Router · 64-Bit SimHash Cache · MCP JSON-RPC 2.0
Automated tests
96

System architecture

  1. Task Complexity Analyzer & Pareto Utility Router (classifier.py / router.py)
  2. Confidence-Gated Cascade & 3-State Circuit Breaker (router.py / health.py)
  3. 64-Bit SimHash Semantic Cache & Budget Governor (cache.py / budget.py)
Three layers from the project’s architecture blueprint

01 / Problem

Why it exists

Multi-agent systems execute dozens of LLM sub-calls per task. Routing every trivial classification or tool-formatting step to a frontier reasoning model wastes 65–90% of token budgets and violates latency SLAs, while single-provider dependence causes workflow crashes during HTTP 429 rate-limit spikes.

02 / Design

How it works

Combine fast pre-execution complexity classification with post-generation response validation so simple steps finish on FAST_LOCAL while hard steps automatically cascade up to FRONTIER_REASONING.

  1. Task Complexity Analyzer & Pareto Utility Router (classifier.py / router.py)

    Scores reasoning depth, code density, and tool complexity to route steps across FAST_LOCAL, BALANCED, and FRONTIER_REASONING endpoints.

  2. Confidence-Gated Cascade & 3-State Circuit Breaker (router.py / health.py)

    Validates lower-tier outputs for JSON/tool validity and confidence >= threshold, escalating on uncertainty and failing over on HTTP 429/5xx faults.

  3. 64-Bit SimHash Semantic Cache & Budget Governor (cache.py / budget.py)

    Eliminates redundant agent sub-queries via locality-sensitive hashing and enforces tenant/workflow token and USD budgets.

Confidence-Gated Speculative Cascade Over Static Model Assignment
Context
Hardcoding a single model per agent either overpays 5–10x on routine steps or fails on unexpected edge cases requiring deep reasoning.
Decision
Combine fast pre-execution complexity classification with post-generation response validation so simple steps finish on FAST_LOCAL while hard steps automatically cascade up to FRONTIER_REASONING.
Trade-offs
Incurs a small cascade retry overhead on the ~10–15% of steps that escalate, but reduces overall multi-agent workflow cost by 79–90% while preserving 100% quality retention.
Verification
Verified across 96 pytest tests, sliding-window rate governor, and Nexus Aegis G2 bridge.
Security and operational constraints
  • Tool-schema fingerprint isolation in SemanticPromptCache ensures cached responses never leak across incompatible tool definitions or tenants.
  • TokenBudgetGovernor enforces hard per-workflow and per-tenant USD/token caps (downgrading tiers at 80% soft cap and halting runaway spend at 100%).
  • Every routing and cascade decision emits a SHA-256 signed RoutingDecisionReceipt for cost and SLA auditing.
  • Endpoints in CircuitState.OPEN are never selected until cooldown expires and a HALF_OPEN probe succeeds.
  • Cascade escalation only accepts a lower-tier response if it passes ResponseValidator (schema validity, non-refusal, and confidence >= min_confidence).
  • Workflows exceeding 100% of their USD or token cap are deterministically blocked before dispatching upstream model calls.

03 / Evidence

How to verify it

Repository-reported / Cost Reduction & Failover Resilience

90.6% Cost Saved · 100% Failover · 96 Tests

0.74 ms P50 routing across 500 multi-agent requests ($3.74 → $0.35 USD) with 100% cascade quality retention and 13/13 HTTP 429 recoveries

Reproduce from the command line
git clone https://github.com/codesphere-u/synapse-router.git
cd synapse-router && pytest -v
python -m synapse_router.cli demo && python -m synapse_router.cli benchmark
View source on GitHubOpen the live demo
Next case study · Agent orchestration SwarmWeaver Coordinate parallel work without competing file edits.