Quantisation: memory by format
The weight format decides how much memory is needed, more surely than the model itself.
Short answer. Moving from FP16 to NVFP4 divides the weight footprint by 4. 28 pages detail each model in each format.
How much memory per format?
The figure covers the weights, runtime margin included, excluding the attention cache.
What does quantisation cost?
- FP16: full precision, the quality reference.
- FP8: negligible loss on most tasks.
- NVFP4: Blackwell format, native FP4 compute.
- Q4: block quantisation, broadly supported.
Method and sources
Footprints are calculated: billion parameters × bytes per parameter of the format × 1.15 runtime margin. They are not measured on hardware. The attention cache sits on top and depends on context and concurrency. Prices are indicative and not contractual. Each detail page carries its sources; the shared conventions are these.
- Bytes per parameter: FP16 2 (16 bits per parameter, hence 2 bytes); FP8 1 (8 bits per parameter (E4M3), hence 1 byte, block scales not counted); NVFP4 0.5 (4 bits per parameter, hence 0.5 byte; 4-bit published weights weigh 0.54 to 0.56 byte per parameter with their scales (DeepSeek V4 Pro 865 GB for 1,599 billion, Kimi K3 1,561 GB for 2,780 billion)); Q4 0.6 (taken here as llama.cpp Q4_K_M, 0.6 byte per parameter including scales; Q4_K_S weighs 0.56, AWQ and GPTQ 0.55). Source: connaissance/faits.yaml, quantifications family, checked on 2026-08-31.
- Runtime margin × 1.15: QDNA operating assumption, not measured: 15% above the weights for activations, buffers and fragmentation.
- “Fits” threshold at 75% of memory: QDNA assumption, not measured, which keeps the remaining quarter for the attention cache and concurrency.