Memory calculator: which LLM fits on which machine?

How much memory does serving an open-weight model take, and which machine does it fit on? This calculator applies the rule of our 210 sizing sheets, adds the key-value cache the sheets do not count, and gives the maximum context each machine can hold.

Published on 6 September 2026 · Version 1.0, 6 September 2026

Short answer. Weights = parameters × bytes of the format × 1.15 execution headroom. A machine "fits" when the weights take less than 75% of its memory, "just fits" between 75 and 100%. The key-value cache adds on top: KiB per token × context × concurrent requests. The maximum context is what remains after the weights, divided by the cost of one token of cache.

Model

Parameters, billions

Weight format

Execution headroom, %

Key-value cache per token

The value is specific to each model: take the served model's, or compute it from its config.json.

Served context, tokens

Concurrent requests

Machine Memory for weights, GB

Across the eight machines of the sheets

Machine	Memory, GB	Verdict	Max context, tokens

The formulas, as applied

Weights alone = parameters (billions) × bytes per parameter of the format. FP16: 2 bytes; FP8: 1; NVFP4: 0.5; Q4: 0.6 including scales. These are format definitions, not measurements.

Weights with headroom = weights alone × 1.15. The 15% headroom for activations, buffers and fragmentation is a QDNA operating assumption, not measured.

Key-value cache = KiB per token × 1,024 × context × concurrent requests. Per-head attention: 2 × layers × KV heads × head dimension × bytes. Latent attention: (latent + RoPE) × layers × bytes.

Verdict: "fits" if weights with headroom ≤ 75% of memory and total ≤ memory; "just fits" if total ≤ memory; "does not fit" otherwise. Maximum context = (memory − weights with headroom) ÷ (KiB per token × 1,024 × requests).

Assumptions and sources, dated

Parameters of the eight models: number announced by the publisher, re-read on 2 September 2026 on config.json and the Hugging Face API (the tensor element count may differ by a few percent; each sheet's note says so).
Memory of the eight machines: manufacturer's data sheet, re-read on 2 September 2026; sum of the GPUs for a server, unified memory for a workstation.
Bytes per parameter: format definitions (connaissance/faits.yaml, quantisation family, verified 31 August 2026). Weights published in 4-bit weigh 0.54 to 0.56 byte with their scales.
Key-value cache: values re-read on config.json in our GLM 5.3, GLM-5.3-Flash, Qwen3.8-27B and Qwen3.8-Flash-Next articles; excluding DSA indexer keys (128 bytes per layer) where present.
Not counted: the embedding model, the reranker, the operating system, runtime memory beyond the headroom, nor speed (see the measurement sheets).

Frequently asked questions

Why do the sizing sheets not count the key-value cache?

Because it depends on two operating choices the sheet does not know: the served context length and the number of concurrent requests. The sheets give the verdict on the weights and reserve a quarter of the memory; this calculator fills that quarter with your values and says how far the context can go.

Is the parameter count enough to compute the weights?

Almost. A repository's real weight depends on the tensor element count and the quantisation scales, which add a few percent that the 15% headroom absorbs. For DeepSeek V4 Pro, 1,599 billion elements published in mixed FP4 and FP8 weigh 865 GB; the NVFP4 calculation gives 800 GB before headroom.

How do I find the cache values for a model missing from the list?

In the repository's config.json: number of layers, key-value heads and head dimension for per-head attention; latent size and RoPE key for latent attention. The "compute from config.json" mode applies the formula shown below, in FP16 or FP8 depending on the runtime.