Quantisation: memory by format
The weight format decides how much memory is needed, more surely than the model itself.
Short answer. Moving from FP16 to NVFP4 divides the weight footprint by 4. 28 pages detail each model in each format.
How much memory per format?
The figure covers the weights, runtime margin included, excluding the attention cache.
What does quantisation cost?
- FP16: full precision, the quality reference.
- FP8: negligible loss on most tasks.
- NVFP4: Blackwell format, native FP4 compute.
- Q4: block quantisation, broadly supported.
Method
Footprints are calculated from the parameter count and the format, with a runtime margin. They are not measured on hardware. The attention cache sits on top and depends on context and concurrency. Prices are indicative and not contractual.