Semantic router

Multi-agent orchestration

The semantic router classifies each request before LiteLLM. Using embedding similarity, it distinguishes confidential content from public content, then routes the request to a local sovereign model or to an external model. This filtering prevents data leakage.

Key points: Embedding-based classification · Confidential or public · Leak prevention · Task-based routing · Ahead of LiteLLM.

Embedding-based classification

The router computes the similarity of each request against reference examples, then categorises it before any model call. The decision relies on the meaning of the text, not on simple keywords.

Confidential or public

The router distinguishes sensitive content from ordinary content. Sensitive content stays on a local sovereign model, while the rest can overflow to an external API when this speeds things up or reduces cost.

Leak prevention

This filtering prevents confidential data from leaving the infrastructure. The organisation's policy applies before execution, not after the fact.

Task-based routing

Beyond sensitivity, the router routes according to the nature of the request, code, reasoning, or long context, to the best-suited model in the catalogue.

Ahead of LiteLLM

The router sits in front of the gateway. The filtering remains transparent to applications, which continue to call a single interface.