Serving a model: which engine for which model
The engine decides throughput, concurrency and the formats accepted.
Short answer. 27 engine and model combinations are documented. Each page gives the minimum memory and the platforms that meet it.
By engine
Dynamo
Dynamo targets distributed serving across several nodes.
Triton Inference Server
Triton Inference Server targets industrial deployment across several models.
vLLM
vLLM targets production serving, throughput and concurrency.
Unsloth
Unsloth targets fine-tuning and quantisation.
llama.cpp
llama.cpp targets workstations and modest hardware.
Method
Footprints are calculated from the parameter count and the format, with a runtime margin. They are not measured on hardware. The attention cache sits on top and depends on context and concurrency. Prices are indicative and not contractual.