Dynamo

Inference Runtime

By QDNA · Published on 5 July 2026 · Updated on 2 September 2026

Dynamo distributes inference for reasoning models across large GPU fleets. It separates the prefill and decode phases and routes requests based on KV cache state, which increases throughput at rack scale.

Key points: Disaggregated · GPU fleet · Prefill and decode separation · Cache-aware routing · Rack scale.

Role

Dynamo distributes inference for reasoning models across large GPU fleets, at rack scale.

How it works

It separates the prefill and decode phases and routes requests based on KV cache state, which increases overall throughput.

Use cases

Dynamo targets high-load deployments and demanding reasoning models, where a single machine is no longer enough.

Integration

It orchestrates multiple vLLM, SGLang or TensorRT-LLM instances and sits above the inference layer for the largest fleets (Dynamo documentation, re-read on 2 September 2026).

License and source code

NVIDIA Dynamo is implemented in Rust, with Python for extensibility, and its source is public. Its LICENSE file places it under the Apache-2.0 licence (re-read on 2 September 2026); GitHub nevertheless returns no standard identifier, because the file opens with a notice about MIT-licensed test data. Read the licence file in the repository before any commercial deployment.

Repository: ai-dynamo/dynamo. Licence and implementation language are re-read from this source by the QDNA fact base. The version number is deliberately not published here: it would go stale between two upstream releases without anything catching it.