vLLM
Inference Runtime
vLLM delivers high-throughput serving for production. It relies on PagedAttention, tensor and pipeline parallelism, automatic prefix cache reuse, and FP8 or NVFP4 quantization. Its interface remains compatible with the OpenAI protocol.
Key points: Production · PagedAttention · Prefix cache · Multi-GPU · FP8 and NVFP4 · OpenAI API.
Role
vLLM delivers high-throughput serving for production. It serves open-weight models behind an interface compatible with the OpenAI protocol.
How it works
It relies on PagedAttention, tensor and pipeline parallelism, automatic prefix cache reuse, and FP8 or NVFP4 quantization (vLLM documentation, re-read on 2 September 2026).
Use cases
vLLM serves multi-user production workloads on GPU servers, where throughput and cost per token matter.
Integration
It is exposed to LiteLLM and to the platform, and is the default inference engine on RTX PRO 6000 and H200 servers.
License and source code
vLLM is published under the Apache-2.0 licence and implemented in Python. The source is public, which means the runtime can be audited, patched and pinned to a known commit rather than trusted on a vendor's word.
Repository: vllm-project/vllm. Licence and implementation language are re-read from this source by the QDNA fact base. The version number is deliberately not published here: it would go stale between two upstream releases without anything catching it.
Which models can vLLM serve?
vLLM targets production serving, throughput and concurrency.
| Model | Billion parameters | Serving page |
|---|---|---|
| DeepSeek V4 | 1,600 | DeepSeek V4 with vLLM |
| GLM 5.2 | 744 | GLM 5.2 with vLLM |
| Kimi K2.7 Code | 1,000 | Kimi K2.7 Code with vLLM |
| Kimi K3 | 2,800 | Kimi K3 with vLLM |
| MiniMax M3 | 428 | MiniMax M3 with vLLM |
| Nemotron 3 Ultra | 550 | Nemotron 3 Ultra with vLLM |
| Qwen 3.8 27B | 27 | Qwen 3.8 27B with vLLM |
See all engine and model combinations.