The principle
Find the cheapest and fastest provider that still meets the required quality floor. Then escalate only if more intelligence is actually needed.
Most AI products send every request to one large model at one company. That is simple, but it is also:
- Expensive: a frontier model answers even simple questions.
- Fragile: when that company has an outage, you have one too.
- Invasive: that company sees your entire history.
Decentralised.si makes each decision per request, on your side. Three rules follow from the principle and appear throughout the code:
Quality is a floor, not a slider
A provider predicted to fall below the task's quality floor is excluded, however cheap it is. If nobody qualifies, the best available provider answers and the result is marked degraded, never passed off as meeting the target.
Every decision is explained
Each candidate gets a score broken into eight normalised components, and the selection comes with plain-language reasons. You can ask for the full decision without running any model.
The device is the source of truth
Conversation memory, the privacy mapping and learned provider statistics live on your machine. Providers receive single, pseudonymised requests. There is no central server in the decision loop.
The system at a glance
There are three layers. Only the first is required. Everything below it is a provider, and Synapse treats them all the same way: an endpoint with a price, an expected latency and quality, and a map of the domains it is good at.
① Your devicerequired · runs locally
② Expert providersany mix · configured in YAML
dsi-node, paid in PAI; reached through DSI Axon, which signs every request they serve③ Fabric servicesroadmap · interfaces in code
Code: DSI-Synapse (layer 1) · apps/gateway and packages/node (layer 2) · src/future.rs (layer 3 interfaces)
Inside Synapse
| Module | Responsibility |
|---|---|
router/ | Classification, difficulty, planning, scoring, escalation ladder, stop rules, synthesis |
context/ | Pseudonymisation, per-provider context compilation, local memory, context deltas |
providers/ | Adapter trait; OpenAI-compatible, Ollama and simulated adapters; registry; health probes |
streams/ | Execution: streaming, hedging, fallback, mid-stream continuation, cancellation |
storage/ | In-memory route cache, SQLite writer thread, outcomes and feedback |
api/ | Local HTTP API with server-sent events |
Life of a request
Every request passes through the same state machine, and each transition is recorded in the task's event log. The first four steps are the fast path. They are pure in-memory work, with no network calls, no database reads and no discovery.
- RECEIVED
Recall and protect
Local conversation history is attached. Identifiers in the prompt and every context item are replaced with placeholders before anything else happens.
- CLASSIFYING
Understand
Domain, difficulty, reasoning profile, freshness and tool needs are derived from the pseudonymised text. The result is cached by a hash of the prompt.
- PLANNING
Set the bar
Mode, difficulty and your limits become a quality floor and target, a latency target, a budget, a deadline and an output-token estimate.
- ROUTING
Choose
Every provider is scored with the latest statistics from the route cache. Constraints are applied, one provider is selected and the rest are ranked as backups.
- STREAMING
Answer, guarded
Context is compiled for the chosen provider and the answer streams to you. If it is slow a backup is started; if it fails the next provider takes over.
- OPTIONAL_ESCALATION
Deepen only if unsure
Uncertainty is measured. Only if it exceeds the mode's threshold is a stronger or independent expert asked, within budget and deadline.
- SYNTHESIZING
Combine
With two answers, the most credible is kept or they are merged.
- COMPLETED · DEGRADED · CANCELLED · FAILED
Record and learn
Time to first token, latency, cost, success and escalation are written to SQLite and update the provider's statistics for that domain.
The client sees it as server-sent events:
$ curl -N localhost:7766/v1/ask -H 'Content-Type: application/json' \ -d '{"prompt": "Explain Black-Scholes implied volatility.", "mode": "balanced"}' event: route {"domain": "finance.quant.derivatives", "difficulty": "easy", "provider": "quant-local", "score": 0.9, "reasons": ["exact domain match (finance.quant.derivatives)", "quality above target (0.96 ≥ 0.88)", "low expected latency (~150 ms to first token)", "within budget (~$0.00019)", "runs locally"], ...} event: token "Implied" event: token " volatility" … event: final {"state": "COMPLETED", "provider": "quant-local", "ttfi_ms": 163, "cost": 0.000058, "escalated": false, "stop_reason": "confident", "hedged": false, "fallbacks": 0, ...}
Code: router/mod.rs (fast path) · streams/manager.rs (execution) · models/outcome.rs (states)
Understanding the request
Domain: a taxonomy with fallback
Expertise is organised as a tree. A provider that is excellent at finance.quant.derivatives is a perfect match for an options question. A finance.quant generalist is a good match, and a generalist is an acceptable last resort.
Classification is layered:
- Layer 1: auditable keyword rules. They take microseconds and always run. A specific domain with evidence of its own inherits its ancestors' evidence, so "Rust" plus generic software words ranks
technology.software.rustabovetechnology.software. Up to three weighted domains are kept. - Layer 2: a small model classifier, used only when rule evidence is weak. It sits behind the
DomainClassifiertrait, so a local model can replace it. It only ever sees the pseudonymised prompt, and never seeslocal_onlyrequests unless it runs locally.
Difficulty: from structure, not meaning
A deterministic estimator adds up signals, and every signal is reported so the decision can be audited:
- Raise difficulty: length, number of domains, high-stakes domains (law, medicine), explicit constraints, need for live data or calculation, multi-step and research-grade language, multi-item comparisons, jurisdiction-specific questions.
- Lower difficulty: short definitional questions.
| Prompt | Difficulty | Why |
|---|---|---|
| What is EBITDA? | easy | short definitional question |
| Write a Rust async TCP server. | medium | code generation |
| Compare three tokenisation structures under Australian securities law. | hard | high-stakes domain, multi-item comparison, jurisdiction, constraint |
| Design a new distributed consensus mechanism and formally reason about failure modes. | extreme | two research-grade cues, three multi-step cues |
Difficulty moves the bar. Harder tasks raise the quality target, and the gap between strong and weak models widens: a provider's predicted quality is pulled down more on hard and extreme tasks.
Code: taxonomy.rs · classify.rs · difficulty.rs · planner.rs
Choosing an expert
Each provider gets eight component scores, each normalised to 0–1, so raw milliseconds and dollars are never added together. The mode decides how much each component counts.
| Component | How it is normalised |
|---|---|
| domain match | competence × taxonomy factor: exact 1.0 · parent 0.85 · broader 0.65 · general 0.40 · narrower 0.70 (configurable) |
| predicted quality | (q − floor) / (1 − floor), with q adjusted for domain fit and difficulty, blended with observed quality |
| latency | 1 / (1 + ttfi / target); the target comes from the mode, and slow providers count double |
| price | 1 / (1 + cost / (5% of budget)), from estimated input and output tokens |
| reliability | completion reliability blended with the observed success rate, × current health |
| tool match | required tools or JSON output supported |
| context affinity | higher if this provider served the conversation last (its prompt cache is warm) |
| privacy match | local providers score highest; remote ones lower as the privacy class tightens |
Modes
80% of the weight follows the mode. The quality share is split 60/40 between domain match and predicted quality. The other 20% is fixed: reliability 10%, tool match 5% and context affinity 5%.
| Mode | Quality | Speed | Cost | Privacy | Also |
|---|---|---|---|---|---|
fast | 0.30 | 0.50 | 0.15 | 0.05 | 300 ms first-token target |
balanced | 0.40 | 0.30 | 0.20 | 0.10 | default |
deep | 0.65 | 0.10 | 0.10 | 0.15 | floor +0.05, lower escalation threshold |
cheap | 0.30 | 0.10 | 0.55 | 0.05 | still never below the floor |
maximum | 0.80 | 0.05 | 0.05 | 0.10 | always asks a second, independent expert |
Constraints before scores
Hard constraints exclude a provider outright: disabled, offline, or not allowed to see this privacy class.
Soft constraints exclude it unless nothing else qualifies: predicted quality below the floor, over budget, or too slow for the deadline. If every provider fails a soft constraint, the best reachable one answers and the task is marked DEGRADED, with the reason.
When the ideal provider is unavailable, the explanation says so. For example: quant-local is offline (best match for finance.quant.derivatives); falling back to general-local.
POST /v1/route returns the whole decision without running a model: the task vector, every candidate's components and reasons, the weights, and the timings.
Code: scorer.rs · api/inspect.rs
What leaves your device
Before anything is transmitted, the prompt and every context item are scrubbed. The scrubber finds:
- email addresses and phone numbers;
- crypto wallets (EVM, Bitcoin, base58);
- IBANs, card numbers (Luhn-checked) and account numbers;
- IP addresses, and URLs that carry tokens or credentials;
- names and other terms you tag as sensitive.
Each is replaced with a stable placeholder. The mapping stays in the daemon's memory: it cannot be serialised, and it is never logged, stored or returned by the API. Answers shown to you are rehydrated as they stream, even when a placeholder is split across chunks.
# You type (privacy: pseudonymised) Email me at you@example.com and analyse this option position. # The provider's server receives — captured on the wire [{"role":"user","content":"Email me at <EMAIL_1> and analyse this option position."}] # You see … reply to you@example.com …
| Privacy class | Behaviour |
|---|---|
public | sent as written |
pseudonymised | default; identifiers replaced before transmission |
confidential | pseudonymised, and only local or explicitly trusted providers may receive it |
local_only | remote providers are never used; if none is local, the request fails rather than leak |
Minimal context, per provider
The context compiler builds a separate context for each provider:
- Removes duplicates.
- Scores each item's relevance, using its domain tag or its word overlap with the question.
- Drops what is unrelated, but keeps the most recent turns of the conversation.
- Fits the rest into that provider's context window, truncating long items head-and-tail.
A cardiology note does not travel with an options question.
Code: privacy.rs · compiler.rs · memory.rs
Streaming without interruption
Hedged requests
A provider that promises a first token in 100 ms but takes 800 ms is detected while it happens. If no token has arrived after the hedge delay, the next-best provider is started. The first provider to produce a token wins, and the other request is cancelled, which closes its connection. The loser's elapsed time is recorded, so a provider that is consistently slower than it claims gets demoted.
These are real measurements from acceptance demo 4, not an illustration.
Fallback and continuation
An error before the first token moves the request to the next backup immediately.
An error after tokens have streamed is handled differently. The partial answer goes to the next provider with an instruction to continue from exactly where it stopped, and you see one uninterrupted reply with a continuing status event.
Health comes from two sources: background probes and real traffic. A refused connection marks a provider offline at once, and offline providers never receive interactive traffic.
Cancellation
DELETE /v1/tasks/:id stops everything: it aborts every provider request, stops escalation, closes the stream and records CANCELLED. Closing the connection does the same, so abandoned work never keeps consuming paid tokens.
Code: streams/manager.rs · providers/health.rs · streams/cancellation.rs
More intelligence, only when needed
After each answer, Synapse estimates uncertainty, from 0 to 1. It combines:
- the provider's self-reported confidence (45%);
- its predicted quality (15%);
- whether the answer is complete (20%);
- how well the provider's domain matched (10%);
- whether required tools were missing (10%).
It climbs the ladder only while uncertainty exceeds the mode's threshold (fast 0.50, cheap 0.45, balanced 0.35, deep 0.25) and the next step fits in the remaining budget and deadline.
The second opinion is chosen for diversity, not similarity: a different provider, preferably from a different model family, and with a different specialty when the question spans domains. Two copies of the same model agreeing is not evidence.
When there are two answers, the most credible one is kept (quality × completeness × confidence). Alternatively, they are merged deterministically or by a configured synthesis model.
The client sees a deepening status, then a new segment, then the result. The maximum mode always takes the R4 step.
Code: escalation.rs · stop.rs · synthesis.rs
Learning from outcomes
Every attempt updates that provider's statistics for the task's domain and overall. The statistics cover:
- successes and failures;
- average time to first token and total latency;
- observed quality and cost.
The scorer reads them from an in-memory route cache. SQLite is written behind it by a dedicated thread, so the request path never waits on disk.
Observations are blended into the configured expectations at weight n / (n + 10), so a handful of samples nudges a provider's score without overturning it. POST /v1/tasks/:id/feedback with a rating of 1, 0 or −1 moves observed quality more strongly than passive signals do.
CREATE TABLE provider_stats ( provider_id TEXT, domain TEXT, successes INTEGER, failures INTEGER, avg_ttfi_ms REAL, avg_total_ms REAL, avg_quality REAL, avg_cost REAL, last_used INTEGER, PRIMARY KEY (provider_id, domain) );
Each task is also persisted as an inspectable record: classification, plan, attempts, events, timing, cost and state. It never contains the prompt or the privacy mapping. Prompts are not logged unless you switch on logging.log_prompts.
Code: storage/cache.rs · storage/sqlite.rs · storage/outcomes.rs
DSI Axon and the network
Not every app can run a daemon, so the same ideas are also available as a hosted service at api.decentralised.si. It runs on Cloudflare Workers, and its code is in the same repository.
Drop-in APIs
Anthropic Messages, OpenAI Chat Completions, Responses and Embeddings, Gemini, and MCP. Change one base URL and keep your SDK. Streaming, tools, vision, files and batches are translated between vendors.
A blind router
Routes on a request's structure and your hints, not its meaning. It strips identity, spreads conversations across providers, and continues interrupted streams on another provider.
Accounts
Bring-your-own keys with envelope encryption, rate and spend limits, workspaces, a full console, and crypto payments.
Open provider network
Anyone can run dsi-node next to an open-weight model (set one up with Docker; see it on the live map). The router probes it, signs its traffic and checks it with canaries. Operators earn PAI for verified work.
Routing order: community nodes first
- Community nodes. If a node meets the request's quality bar, the network serves it. The bar comes from the reasoning level, and a node's declared expertise in the topic (the client's domain hint) counts toward it. Specialists answer in their field, generalist nodes answer everyday questions, and the session key spreads conversations across equal nodes.
- Hosted open models (fallback). Open-weight models on Cloudflare Workers AI answer when no node meets the bar yet, and when nodes fail mid-request. The chat labels them "Cloudflare-hosted fallback", and they get no decentralisation credit.
- Your vendor keys. Hard requests (reasoning level 3) go to the strongest model you have connected, with nodes kept as failover. The
quality,cheapestandfastestmodes compare everything on their own terms;freeanddecentralised_onlyuse nodes only.
The two fit together naturally. Axon speaks the OpenAI-compatible API, so Synapse can use it as one more provider, next to your local models and your own keys. Synapse then decides when Axon is the right expert, and your memory and privacy mapping stay on your machine.
Code: apps/gateway · packages/core/src/router.ts · continuity.ts · packages/node · packages/harness · whitepaper
Measured performance
These routing-overhead figures come from the benchmark harness on a laptop, in a release build: classification, difficulty and scoring, excluding model inference.
The targets were under 2 ms for scoring 100 providers, under 5 ms for a route-cache lookup and under 10 ms for planning.
The benchmark also compares strategies over a mixed workload with simulated providers:
| Strategy | First token p50 | Cost / 1k requests | Simulated quality |
|---|---|---|---|
| always the general model | 87 ms | $0.013 | 0.830 |
| always the frontier model | 451 ms | $0.791 | 0.915 |
| Synapse adaptive | 161 ms | $0.064 | 0.923 |
Latency, cost and quality are simulated from each provider's configuration. The test prompts were written alongside the classifier's rules. The result shows that routing sends work where the configured expertise is; it is not an evaluation of real model answers. Run cargo run --release --bin benchmark and publish your own numbers.
What is built
| Component | Where | Status |
|---|---|---|
| DSI Synapse, the local router (Rust) | DSI-Synapse | preview tested; binaries for macOS, Linux, Windows |
| DSI Axon, the hosted blind router: APIs, limits, billing | apps/gateway, packages/core | live |
| Chat app, console, wallet | web | live |
| dsi harness (local memory, MCP server) | packages/harness | implemented download |
| Live network map (by city) | /network, /api/network/map | live |
| dsi-node provider agent | packages/node | implemented download |
| PAI ledger | packages/pai | off-chain |
| PAI contract | contracts/ | reference, unaudited, not deployed |
| R3 tools / retrieval, R5 verifier | synapse/src/future.rs | interfaces |
| Global discovery, shared reputation, settlement, TEE nodes | — | roadmap |
Run it yourself
Install a prebuilt binary for macOS (Apple silicon or Intel), Linux (x86_64 or arm64) or Windows. The download page has one-line installers, direct downloads, checksums and background-service setup.
curl -fsSL https://decentralised.si/install/synapse.sh | sh # macOS, Linux irm https://decentralised.si/install/synapse.ps1 | iex # Windows PowerShell synapse
Or build from source with a Rust toolchain (Docker works too). The default configuration uses three simulated experts, so no GPU or API key is required to see routing work.
git clone https://github.com/Decentralised-si/DSI-Synapse cd DSI-Synapse cargo run --release # listens on 127.0.0.1:7766 curl -N localhost:7766/v1/ask -H 'Content-Type: application/json' \ -d '{"prompt": "Explain Black-Scholes implied volatility.", "mode": "balanced"}' curl localhost:7766/v1/route -H 'Content-Type: application/json' \ -d '{"prompt": "Write a Rust async TCP server."}' # the decision, no inference cargo test # tests: routing, fallback, privacy, escalation, streaming cargo run --release --bin benchmark # overhead, TTFI, cost; CSV output
To use real models, point config/providers.yaml at any OpenAI-compatible server (vLLM, llama.cpp, LM Studio, a hosted API or DSI Axon) or at Ollama. Declare what each one is good at:
providers:
- id: quant-local
endpoint: http://127.0.0.1:8101/v1
adapter: openai # openai | ollama | mock
model: quant-specialist
domains: { finance.quant: 0.90, finance.quant.derivatives: 0.97 }
price: { input_per_million: 0.20, output_per_million: 0.60 }
expected_ttfi_ms: 150
expected_quality: 0.95
The daemon binds to 127.0.0.1 and refuses any other address unless an API token is set. Provider keys are read from environment variables named in the config, never written into it.
Decentralised-si/DSI-Synapse
DSI Synapse, the router on your device (Rust): tests, benchmark, Docker image and release builds.
src/ · tests/ · config/
Decentralised-si/DSI-AXON
DSI Axon, the blind router (TypeScript), dsi-node, the dsi harness, the PAI ledger and contract, and the whitepaper.
apps/gateway · packages/* · contracts/
Decentralised-si/web
This website, the chat app, the console and the wallet (React + Privy on Cloudflare).
public/ · app/ · src/
Issues and pull requests are welcome in all three repositories. Good first areas are a learned difficulty estimator, a local embedding classifier, new provider adapters, and the R3 and R5 rungs.