vLLM
vLLM delivers high-throughput serving for production.
Triton Inference Server
Triton serves models from multiple frameworks on the same infrastructure.
Dynamo
Dynamo distributes inference for reasoning models across large GPU fleets.
llama.cpp
llama.cpp stays lightweight.
Unsloth
Unsloth speeds up fine-tuning by a factor of two, according to its publisher, while cutting the memory required, through the LoRA and QLoRA methods.
dwarfstar
dwarfstar is the sovereign stack developed by QDNA.
Which engine should you choose?
The engine does not change what a model can do. It decides throughput, how many concurrent requests the machine sustains, and which weight formats are accepted. The same model served by two engines needs the same memory for its weights, but does not deliver the same service.
| Engine | What it targets | Documented models |
|---|---|---|
| vLLM | production serving, throughput and concurrency | 7 |
| Triton Inference Server | industrial deployment across several models | 7 |
| Dynamo | distributed serving across several nodes | 7 |
| llama.cpp | workstations and modest hardware | 1 |
| Unsloth | fine-tuning and quantisation | 5 |
How much memory per model?
The weight footprint depends on the model and the format, not on the engine. In NVFP4 it ranges from 16 GB for the most compact to 1,610 GB for the largest open model in the catalogue. The attention cache sits on top and grows with the context served.
Combination by combination in serving a model, machine by machine in sizing.
Documented combinations
- DeepSeek V4 with Dynamo
- DeepSeek V4 with Triton Inference Server
- DeepSeek V4 with vLLM
- GLM 5.2 with Dynamo
- GLM 5.2 with Triton Inference Server
- GLM 5.2 with Unsloth
- GLM 5.2 with vLLM
- Kimi K2.7 Code with Dynamo
- Kimi K2.7 Code with Triton Inference Server
- Kimi K2.7 Code with Unsloth
See also the quantisation comparison, which decides memory more surely than the model itself.