LLM VRAM calculator: which model fits on which machine?
How much memory does it take to serve an open-weight model, and which machine can hold it? This calculator, also called an LLM sizer or GPU sizer, counts the published weights, the key-value cache and the state of recurrent layers. It checks, machine by machine, the memory of each GPU, the speed per user and the time to first token.
Size inference and choose the machine
Recommended configuration
QLoRA
2. Weight format
3. Key-value cache precision
Rank-64 LoRA on the linear layers, excluding the routed experts of an MoE (attention, shared experts and dense layers); adapters in BF16 rather than FP16.
A batch of 1 with gradient accumulation is usually enough.
Advanced assumptions
A gateway response cache frees no GPU memory: it only reduces the prefill demand.
These estimates are computed from vendor datasheets and config.json files, then calibrated on 107 published measurements, 61 of them on racks, GPU and TPU (SemiAnalysis InferenceX); 100 fall within 30%. They do not replace a measurement on your workload, with your engine, your prompts and your users.
No machine matches the current filters.
Maximum context per machine
Maximum context a single user can open once the weights are loaded, on eight reference machines: NVFP4 on hardware that runs it, FP8 elsewhere, with an 8-bit cache. Several users share this token budget.
| Model | Lenovo ThinkStation PGX GB10 | RTX 5090 Station (custom build) | Workstation 1x NVIDIA RTX PRO 6000 Blackwell (96 GB) | Apple Mac Studio M5 Ultra (512 GB unified, 1,200 GB/s) | Supermicro Super AI Station (ARS-511GD-NB-LCC) | Server 8x NVIDIA RTX PRO 6000 Blackwell Server Edition (768 GB) | NVIDIA DGX H200 (8x H200) | NVIDIA DGX B300 (8x B300) |
|---|---|---|---|---|---|---|---|---|
| Kimi K3 | does not fit | does not fit | does not fit | does not fit | does not fit | does not fit | does not fit | 1,048,576 |
| DeepSeek V4 Pro 0813 | does not fit | does not fit | does not fit | does not fit | does not fit | does not fit | does not fit | 1,048,576 |
| DeepSeek V4.1 Flash 1017 | does not fit | does not fit | does not fit | does not fit | 1,048,576 | 1,048,576 | 1,048,576 | 1,048,576 |
| GLM 5.3 | does not fit | does not fit | does not fit | does not fit | 1,048,576 | 632,851 | 781,218 | 1,048,576 |
| GLM 5.3 Flash | does not fit | does not fit | does not fit | 1,048,576 | 1,048,576 | 1,048,576 | 1,048,576 | 1,048,576 |
| Mistral Large 4 | does not fit | does not fit | does not fit | does not fit | cache not published | cache not published | cache not published | cache not published |
| MiniMax M3 | does not fit | does not fit | does not fit | does not fit | 1,048,576 | 1,048,576 | 1,048,576 | 1,048,576 |
| Qwen 3.5 | does not fit | does not fit | does not fit | 895,254 (YaRN) | 1,010,000 (YaRN) | 1,010,000 (YaRN) | 1,010,000 (YaRN) | 1,010,000 (YaRN) |
| Qwen 3.8 Flash Next | does not fit | does not fit | does not fit | 1,000,000 (YaRN) | 1,000,000 (YaRN) | 1,000,000 (YaRN) | 1,000,000 (YaRN) | 1,000,000 (YaRN) |
| DeepSeek-V3 | does not fit | does not fit | does not fit | does not fit | 163,840 (YaRN) | 163,840 (YaRN) | 163,840 (YaRN) | 163,840 (YaRN) |
| DeepSeek Coder V2 | does not fit | does not fit | does not fit | 163,840 (YaRN) | 163,840 (YaRN) | 163,840 (YaRN) | 163,840 (YaRN) | 163,840 (YaRN) |
| Llama 3.1 405B | does not fit | does not fit | does not fit | 8,761 | 46,316 | 131,072 | 131,072 | 131,072 |
| Nemotron-4 340B | does not fit | does not fit | does not fit | 4,096 | 4,096 | 4,096 | 4,096 | 4,096 |
| Mistral Large 2 | 131,072 | does not fit | 122,199 | 131,072 | 131,072 | 131,072 | 131,072 | 131,072 |
| Llama 3.3 70B | 131,072 | does not fit | 131,072 | 131,072 | 131,072 | 131,072 | 131,072 | 131,072 |
| Qwen 2.5 72B | 131,072 (YaRN) | does not fit | 131,072 (YaRN) | 131,072 (YaRN) | 131,072 (YaRN) | 131,072 (YaRN) | 131,072 (YaRN) | 131,072 (YaRN) |
| Qwen 2.5 Coder 32B | 131,072 (YaRN) | 63,008 (YaRN) | 131,072 (YaRN) | 131,072 (YaRN) | 131,072 (YaRN) | 131,072 (YaRN) | 131,072 (YaRN) | 131,072 (YaRN) |
| Qwen 3.8 27B | 1,000,000 (YaRN) | 209,826 | 1,000,000 (YaRN) | 1,000,000 (YaRN) | 1,000,000 (YaRN) | 1,000,000 (YaRN) | 1,000,000 (YaRN) | 1,000,000 (YaRN) |
| Devstral Small 24B | 131,072 | 131,072 | 131,072 | 131,072 | 131,072 | 131,072 | 131,072 | 131,072 |
| Llama 3.1 8B | 131,072 | 131,072 | 131,072 | 131,072 | 131,072 | 131,072 | 131,072 | 131,072 |
Monthly capacity per model and machine
This table gives, for each model, the millions of tokens a machine generates in a month at full load, over 730 hours. Full load gathers as many users as possible at 30 tokens per second each, with a first token under one second.
Requests are 8,192 tokens long, and weights are NVFP4 on Blackwell hardware, FP8 elsewhere. At 30% activity, the useful volume is 30% of these values: that is the one the cost per million tokens uses.
| Model | Lenovo ThinkStation PGX GB10 | RTX 5090 Station (custom build) | Workstation 1x NVIDIA RTX PRO 6000 Blackwell (96 GB) | Apple Mac Studio M5 Ultra (512 GB unified, 1,200 GB/s) | Supermicro Super AI Station (ARS-511GD-NB-LCC) | Server 8x NVIDIA RTX PRO 6000 Blackwell Server Edition (768 GB) | NVIDIA DGX H200 (8x H200) | NVIDIA DGX B300 (8x B300) | NVIDIA GB300 NVL72 (complete rack, 72 Blackwell Ultra GPUs) |
|---|---|---|---|---|---|---|---|---|---|
| Kimi K3 | does not fit | does not fit | does not fit | does not fit | does not fit | does not fit | does not fit | 1,167 | 10,446 |
| DeepSeek V4 Pro 0813 | does not fit | does not fit | does not fit | does not fit | does not fit | does not fit | does not fit | 5,667 | 123,333 |
| DeepSeek V4.1 Flash 1017 | does not fit | does not fit | does not fit | does not fit | target not met | 2,510 | 7,186 | 91,685 | 1,092,701 |
| GLM 5.3 | does not fit | does not fit | does not fit | does not fit | target not met | 313 | 1,153 | 7,814 | 142,835 |
| GLM 5.3 Flash | does not fit | does not fit | does not fit | target not met | 1,952 | 2,563 | 8,029 | 54,965 | 564,872 |
| Mistral Large 4 | does not fit | does not fit | does not fit | does not fit | cache not published | cache not published | cache not published | cache not published | cache not published |
| MiniMax M3 | does not fit | does not fit | does not fit | does not fit | target not met | 1,102 | 3,411 | 31,763 | 351,803 |
| Qwen 3.5 | does not fit | does not fit | does not fit | target not met | target not met | 1,709 | 4,995 | 33,159 | 391,386 |
| Qwen 3.8 Flash Next | does not fit | does not fit | does not fit | target not met | 10,862 | 10,875 | 30,290 | 123,149 | 1,157,520 |
| DeepSeek-V3 | does not fit | does not fit | does not fit | does not fit | target not met | 488 | 1,415 | 8,309 | 156,437 |
| DeepSeek Coder V2 | does not fit | does not fit | does not fit | target not met | 1,945 | 1,229 | 4,004 | 27,928 | 278,228 |
| Llama 3.1 405B | does not fit | does not fit | does not fit | target not met | target not met | target not met | 261 | 4,818 | 41,506 |
| Nemotron-4 340B | does not fit | does not fit | does not fit | target not met | target not met | 120 | 1,488 | 12,695 | 109,853 |
| Mistral Large 2 | target not met | does not fit | target not met | target not met | 1,091 | 1,569 | 2,996 | 15,191 | 131,954 |
| Llama 3.3 70B | target not met | does not fit | target not met | target not met | 2,248 | 2,965 | 5,475 | 23,155 | 201,548 |
| Qwen 2.5 72B | target not met | does not fit | target not met | target not met | 2,191 | 2,903 | 5,354 | 22,886 | 199,744 |
| Qwen 2.5 Coder 32B | target not met | 216 | 431 | target not met | 4,402 | 5,964 | 11,409 | 39,335 | 346,255 |
| Qwen 3.8 27B | target not met | 658 | 781 | target not met | 7,596 | 8,059 | 15,339 | 53,366 | 471,494 |
| Devstral Small 24B | target not met | 826 | 968 | target not met | 7,366 | 10,101 | 17,962 | 64,536 | 566,763 |
| Llama 3.1 8B | target not met | 2,002 | 2,375 | target not met | 12,208 | 18,367 | 39,530 | 104,332 | 925,766 |
The formulas, as applied
The calculator applies the formulas below, with no correction other than the ones named. Every result is a calculation from vendor datasheets and config.json files, calibrated on published measurements, and does not replace a measurement on your real workload.
- Weights: size of the published checkpoint for the format, otherwise parameters per tensor class × bytes of the format (2 in BF16, 1 in FP8, 0.53 in INT4 with its scales, 0.5625 in NVFP4, 0.606 in Q4_K_M), head and embeddings in BF16 (Q6_K, 0.82 byte, for GGUF).
- GGUF Q4_K_M: size of the files published on Hugging Face, read on 8 October 2026; loaded by llama.cpp, Ollama and TGI, and experimentally by vLLM and SGLang on NVIDIA only. llama.cpp splits layers across GPUs, without adding up their bandwidth for one user. DeepSeek V4.1 Flash 1017 only has GGUF files for a llama.cpp fork, Mistral Large 4 none.
- Key-value cache: per token and layer: 2 × KV heads × dimension × bytes for per-head attention, latent rank + RoPE for latent attention, published bytes for DeepSeek V4; times context and users, a shared prefix counted once when the engine shares it.
- Recurrent state: linear layers (Gated DeltaNet, KDA): heads × dk × dv × 4 bytes per request, constant with context.
- Usable memory: per GPU: 92% of memory minus 2.5 GiB; Apple: 75% of unified memory; Grace: HBM only, LPDDR5X as spill space.
- Decode: one step reads the active weights (the union of experts used by the batch) and every user's cache; duration = max(bytes ÷ (bandwidth × efficiency), compute ÷ (dense PFLOPS × efficiency)) + fixed per-layer latency; speed per user = 1 ÷ duration, times the engine family factor.
- Prefill: in chunks: 2 × active parameters × tokens, plus causal attention, at the format's rate on the hardware; mean and worst-case first token from a closed loop of users who read the answer then send a new request.
- Status: qualified when memory suffices, speed meets the target and the first token meets the delay, without hidden tolerance; partial when a criterion fails; out of range when memory is short; undetermined when the calculation is not possible.
- Full load: the largest number of simultaneous users that still meets the targeted speed, first token and memory, found by search; a 72-GPU rack or a TPU pod is judged at that number, not at a handful of users. On TPU, experts are spread in groups of 4 chips, enlarged only when the model does not fit, as in published deployments.
- Cost: (price excl. VAT ÷ 26,280 h + power × €0.1624 per kWh) ÷ tokens generated per hour at the chosen activity rate, with the users shown and at full load.
Assumptions and sources, dated
The assumptions are shared by our tools and dated: electricity at €0.1624 excl. VAT per kWh, French professional Tarif Bleu of 1 August 2026; amortisation over thirty-six months of 730 hours; activity 30% of the hours. Prices excluding VAT are dated surveys or indicative ranges, raised by 30% in October 2026 to follow the observed hardware price increase.
These estimates are computed from vendor datasheets and config.json files, then calibrated on 107 published measurements, 61 of them on racks, GPU and TPU (SemiAnalysis InferenceX); 100 fall within 30%. They do not replace a measurement on your workload, with your engine, your prompts and your users.
Boxes stacked over RDMA
GB10 boxes stack in pairs over a direct link, then by three, four or eight behind an RDMA switch, at 200 Gb/s per box. DGX Station GB300 machines link two at most, directly, over two 400 Gb/s cables. Only these published clusters are offered.
For each stackable machine it evaluates every quantity and keeps the smallest one reaching the best verdict without memory spill. Capacities, price and power are multiplied, network included. Link time is added at every step, and the fixed per-layer cost is fitted on measured clusters: GB10 by ai-muninn and the NVIDIA forum, GB300 by StorageReview.
- NVIDIA, Connecting DGX Spark Systems into a Cluster
- NVIDIA, Multi Sparks Through a Switch playbook
- NVIDIA, DGX Station datasheet: two Stations at most
- NVIDIA, Connect Two DGX Stations playbook
- ai-muninn, two DGX Sparks at TP2, 12/09/2026
- NVIDIA forum, Qwen 3.8 27B on 1, 2 and 4 DGX Sparks
- StorageReview, two DGX Station GB300, 05/10/2026
- StorageReview, two DGX Sparks in pipeline
- LLMKube, three DGX Sparks at TP3
- NVIDIA forum, four DGX Sparks at TP4
- NVIDIA forum, eight DGX Sparks at TP8
- OpenZeka, eight DGX Sparks on a MikroTik CRS804
- MikroTik, CRS804 DDQ: 2 × 200G breakout and 123 W
- Switch price, surveyed 12/10/2026
- NVIDIA cable price for DGX Spark
- 400G to 2 × 200G breakout cable price
- Amphenol 400G QSFP112 cable price
- ECB rate of 12/10/2026: $1.1181
Catalogue of the 66 machines
This table repeats the calculator data in a form readable without JavaScript: memory, bandwidth, dense BF16, FP8 and FP4 compute from vendor datasheets, and the power and price excluding VAT used for the cost. Structured-sparsity figures, twice as high, are left aside.
| Machine | GPU | Memory | Bandwidth | Dense BF16 (PF) | Dense FP8 (PF) | Dense FP4 (PF) | Power used | Price used, excl. VAT | Source |
|---|---|---|---|---|---|---|---|---|---|
| AI Mini-PC AMD Ryzen AI Max+ 395 (Strix Halo 128 GB) | AMD Radeon 8060S (40 RDNA 3.5 compute units) | 128 GB | 0.26 TB/s | 0.06 | none | none | 180 W | €3,333 excl. VAT | datasheet |
| AI workstation 2x NVIDIA RTX 4090 (48 GB) | 2x NVIDIA GeForce RTX 4090 (Ada Lovelace) | 48.0 GB | 2.02 TB/s | 0.33 | 0.66 | none | 1,250 W | €4,167 excl. VAT | datasheet |
| Apple Mac Studio M2 Ultra (192 GB unified, used) | Apple M2 Ultra, 76-core GPU | 192 GB | 0.80 TB/s | 0.03 | none | none | 295 W | €4,200 excl. VAT | datasheet |
| Tenstorrent Wormhole n300 workstation (n300 card and host) | n300 card: 2 Wormhole chips with Tensix cores (RISC-V) | 24.0 GB | 0.58 TB/s | 0.13 | 0.47 | none | 650 W | €4,300 excl. VAT | datasheet |
| Mobile workstation RTX PRO 5000 Blackwell (24 GB) | 1x NVIDIA RTX PRO 5000 Blackwell laptop GPU | 24.0 GB | 0.90 TB/s | 0.23 | 0.46 | 0.91 | 230 W | €5,208 excl. VAT | datasheet |
| Apple Mac Studio M5 Ultra (96 GB unified, 1,200 GB/s) | Apple M5 Ultra, 64-core GPU with neural accelerators (base chip) | 96.0 GB | 1.20 TB/s | 0.08 | none | none | 385 W | €5,499 excl. VAT | datasheet |
| Lenovo ThinkStation PGX GB10 | NVIDIA GB10 Grace Blackwell | 128 GB | 0.27 TB/s | 0.12 | 0.25 | 0.50 | 240 W | €6,500 excl. VAT | datasheet |
| RTX 5090 Station (custom build) | 1x NVIDIA GeForce RTX 5090 (Blackwell GB202) | 32.0 GB | 1.79 TB/s | 0.21 | 0.42 | 1.68 | 925 W | €6,500 excl. VAT | datasheet |
| HP ZGX Nano G1n AI Station | NVIDIA GB10 Grace Blackwell | 128 GB | 0.27 TB/s | 0.12 | 0.25 | 0.50 | 240 W | €7,100 excl. VAT | datasheet |
| Gigabyte AI TOP ATOM | NVIDIA GB10 Grace Blackwell | 128 GB | 0.27 TB/s | 0.12 | 0.25 | 0.50 | 240 W | €7,100 excl. VAT | datasheet |
| Dell Pro Precision with GB10 (formerly Pro Max, DGX Spark) | NVIDIA GB10 Grace Blackwell | 128 GB | 0.27 TB/s | 0.12 | 0.25 | 0.50 | 280 W | €7,400 excl. VAT | datasheet |
| ASUS Ascent GX10 | NVIDIA GB10 Grace Blackwell | 128 GB | 0.27 TB/s | 0.12 | 0.25 | 0.50 | 240 W | €8,300 excl. VAT | datasheet |
| Cluster of 2 Mac Studio M2 Ultra (384 GB, exo or llama.cpp RPC) | 2x Apple M2 Ultra 192 GB (76 GPU cores each) | 384 GB | 1.60 TB/s | 0.05 | none | none | 590 W | €8,400 excl. VAT | datasheet |
| AI workstation 4x NVIDIA RTX 4090 (96 GB) | 4x NVIDIA GeForce RTX 4090 (Ada Lovelace) | 96.0 GB | 4.03 TB/s | 0.66 | 1.32 | none | 2,150 W | €9,167 excl. VAT | datasheet |
| Workstation 1x NVIDIA RTX PRO 5000 Blackwell (48 GB) | 1x NVIDIA RTX PRO 5000 Blackwell (GB202) | 48.0 GB | 1.34 TB/s | 0.26 | 0.52 | 1.03 | 650 W | €9,600 excl. VAT | datasheet |
| Apple Mac Studio M5 Ultra (256 GB unified, 1,200 GB/s) | Apple M5 Ultra, 80-core GPU with neural accelerators | 256 GB | 1.20 TB/s | 0.10 | none | none | 385 W | €10,358 excl. VAT | datasheet |
| NVIDIA RTX PRO 6000 Blackwell Max-Q Edition (96 GB) | 1x NVIDIA RTX PRO 6000 Blackwell Max-Q (GB202) | 96.0 GB | 1.79 TB/s | 0.44 | 0.88 | 1.76 | 650 W | €20,000 excl. VAT | datasheet |
| Workstation 1x NVIDIA RTX PRO 6000 Blackwell (96 GB) | 1x NVIDIA RTX PRO 6000 Blackwell (GB202) | 96.0 GB | 1.79 TB/s | 0.50 | 1.01 | 2.02 | 950 W | €23,000 excl. VAT | datasheet |
| Supermicro SYS-221GE-NR (2x L40S) | 2x NVIDIA L40S PCIe 48 GB | 96.0 GB | 1.73 TB/s | 0.72 | 1.47 | none | 1,500 W | €33,000 excl. VAT | datasheet |
| Supermicro ARS-111GL-NHR (1x GH200 Grace Hopper) | NVIDIA GH200 Grace Hopper Superchip | 576 GB | 4.00 TB/s | 0.99 | 1.98 | none | 1,300 W | €48,000 excl. VAT | datasheet |
| Server 4x NVIDIA RTX PRO 5000 Blackwell (192 GB) | 4x NVIDIA RTX PRO 5000 Blackwell 48 GB (GB202) | 192 GB | 5.38 TB/s | 1.03 | 2.06 | 4.13 | 2,000 W | €50,000 excl. VAT | datasheet |
| Workstation 4x NVIDIA RTX 6000 Ada | 4x NVIDIA RTX 6000 Ada Generation (PCIe) | 192 GB | 3.84 TB/s | 1.46 | 2.91 | none | 1,550 W | €55,000 excl. VAT | datasheet |
| Dell PowerEdge R750xa (4x A100 PCIe) | 4x NVIDIA A100 PCIe 80 GB | 320 GB | 7.74 TB/s | 1.25 | none | none | 2,000 W | €59,000 excl. VAT | datasheet |
| NVIDIA DGX Station A100 320 GB (4x A100 80 GB) | 4x NVIDIA A100 SXM4 80 GB (HBM2e) | 320 GB | 8.16 TB/s | 1.25 | none | none | 1,500 W | €76,000 excl. VAT | datasheet |
| Server 8x NVIDIA RTX PRO 5000 Blackwell (384 GB) | 8x NVIDIA RTX PRO 5000 Blackwell 48 GB (GB202) | 384 GB | 10.8 TB/s | 2.06 | 4.13 | 8.26 | 3,900 W | €96,000 excl. VAT | datasheet |
| Server 4x NVIDIA RTX PRO 6000 Blackwell Server Edition (384 GB) | 4x NVIDIA RTX PRO 6000 Blackwell Server Edition (96 GB each) | 384 GB | 6.39 TB/s | 2.00 | 4.00 | 8.00 | 3,200 W | €99,000 excl. VAT | datasheet |
| Supermicro Super AI Station (ARS-511GD-NB-LCC) | NVIDIA GB300 Blackwell Ultra Desktop | 748 GB | 7.10 TB/s | 2.50 | 5.00 | 15 | 1,600 W | €114,000 excl. VAT | datasheet |
| MSI XpertStation WS300 (formerly CT60-S8060) | NVIDIA GB300 Blackwell Ultra Desktop | 748 GB | 7.10 TB/s | 2.50 | 5.00 | 15 | 1,600 W | €117,000 excl. VAT | datasheet |
| Exxact Valence VWS-158270643 (integrator, MSI-based) | NVIDIA GB300 Blackwell Ultra Desktop | 748 GB | 7.10 TB/s | 2.50 | 5.00 | 15 | 1,600 W | €124,000 excl. VAT | datasheet |
| BIZON GB300 (integrator) | NVIDIA GB300 Blackwell Ultra Desktop | 748 GB | 7.10 TB/s | 2.50 | 5.00 | 15 | 1,600 W | €128,000 excl. VAT | datasheet |
| Gigabyte W775-V10-L01 | NVIDIA GB300 Blackwell Ultra Desktop | 748 GB | 7.10 TB/s | 2.50 | 5.00 | 15 | 1,600 W | €129,000 excl. VAT | datasheet |
| HP ZGX Fury G1n | NVIDIA GB300 Blackwell Ultra Desktop | 748 GB | 7.10 TB/s | 2.50 | 5.00 | 15 | 1,600 W | €143,000 excl. VAT | datasheet |
| ASUS ExpertCenter Pro ET900N G3 | NVIDIA GB300 Blackwell Ultra Desktop | 748 GB | 7.10 TB/s | 2.50 | 5.00 | 15 | 1,600 W | €144,000 excl. VAT | datasheet |
| NVIDIA DGX A100 640 GB (8x A100 80 GB) | 8x NVIDIA A100 SXM4 80 GB | 640 GB | 16.3 TB/s | 2.50 | none | none | 6,500 W | €150,000 excl. VAT | datasheet |
| Dell PowerEdge R760xa (4x H100 PCIe) | 4x NVIDIA H100 PCIe 80 GB (350 W) | 320 GB | 8.00 TB/s | 3.03 | 6.05 | none | 2,200 W | €179,000 excl. VAT | datasheet |
| HPE ProLiant DL380a Gen11 (4x H100 PCIe 80 GB) | 4x NVIDIA H100 PCIe 80 GB (HBM2e) | 320 GB | 8.00 TB/s | 3.03 | 6.05 | none | 2,200 W | €189,000 excl. VAT | datasheet |
| Server 8x NVIDIA RTX PRO 6000 Blackwell Server Edition (768 GB) | 8x NVIDIA RTX PRO 6000 Blackwell Server Edition (96 GB each) | 768 GB | 12.8 TB/s | 4.00 | 8.00 | 16 | 6,300 W | €192,000 excl. VAT | datasheet |
| Supermicro SYS-421GE-TNRT3 (4x H100 NVL 94 GB) | 4x NVIDIA H100 NVL 94 GB (2 NVLink-bridged pairs) | 376 GB | 15.6 TB/s | 3.34 | 6.68 | none | 2,400 W | €220,000 excl. VAT | datasheet |
| Dell Pro Precision with GB300 (formerly Pro Max, FCT6263) | NVIDIA GB300 Blackwell Ultra Desktop | 748 GB | 7.10 TB/s | 2.50 | 5.00 | 15 | 1,600 W | €225,000 excl. VAT | datasheet |
| SambaNova SN50 (8-RDU node) | 8x SambaNova SN50 RDUs (reconfigurable dataflow units) | 512 GB | 14.7 TB/s | 13 | 26 | none | 10,000 W | €263,000 excl. VAT | datasheet |
| Supermicro AS-8125GS-TNMR2 (8x MI300X) | 8x AMD Instinct MI300X OAM | 1,536 GB | 42.4 TB/s | 10 | 21 | none | 10,600 W | €277,000 excl. VAT | datasheet |
| Dell PowerEdge XE9680 (8x AMD MI300X) | 8x AMD Instinct MI300X OAM | 1,536 GB | 42.4 TB/s | 10 | 21 | none | 10,600 W | €286,000 excl. VAT | datasheet |
| Supermicro AS-4125GS-TNRT1 (8x H100 PCIe) | 8x NVIDIA H100 PCIe 80 GB (350 W) | 640 GB | 16.0 TB/s | 6.05 | 12 | none | 4,300 W | €293,000 excl. VAT | datasheet |
| ASUS ESC N8-E11 (8x H100 HGX) | 8x NVIDIA H100 SXM5 | 640 GB | 26.8 TB/s | 7.92 | 16 | none | 10,200 W | €319,000 excl. VAT | datasheet |
| Gigabyte G593-SD0 (8x H100 HGX) | 8x NVIDIA H100 SXM5 | 640 GB | 26.8 TB/s | 7.92 | 16 | none | 10,200 W | €325,000 excl. VAT | datasheet |
| Supermicro SYS-821GE-TNHR (8x H100) | 8x NVIDIA H100 SXM5 | 640 GB | 26.8 TB/s | 7.92 | 16 | none | 10,200 W | €335,000 excl. VAT | datasheet |
| Supermicro AS-8126GS-TNMR (8x AMD MI325X) | 8x AMD Instinct MI325X OAM (256 GB HBM3e each) | 2,048 GB | 48.0 TB/s | 10 | 21 | none | 12,600 W | €335,000 excl. VAT | datasheet |
| Dell PowerEdge XE9680 (8x H100 SXM5) | 8x NVIDIA H100 SXM5 | 640 GB | 26.8 TB/s | 7.92 | 16 | none | 10,200 W | €361,000 excl. VAT | datasheet |
| 8x AMD Instinct MI355X OAM Server (CDNA 4) | 8x AMD Instinct MI355X OAM (CDNA 4) | 2,304 GB | 64.0 TB/s | 20 | 40 | 81 | 15,800 W | €380,000 excl. VAT | datasheet |
| Lenovo ThinkSystem SC777 V4 Neptune (GB200 NVL4) | 4x NVIDIA B200 (GB200 NVL4, 186 GB each) | 744 GB | 32.0 TB/s | 10 | 20 | 40 | 5,800 W | €393,000 excl. VAT | datasheet |
| Supermicro SYS-821GE-TNHR-B100 (8x B100, never marketed) | 8x NVIDIA Blackwell B100 SXM (192 GB HBM3e, preliminary specifications) | 1,536 GB | 64.0 TB/s | 14 | 28 | 56 | 10,200 W | €410,000 excl. VAT | datasheet |
| NVIDIA DGX H200 (8x H200) | 8x NVIDIA H200 SXM5 | 1,128 GB | 38.4 TB/s | 7.92 | 16 | none | 10,200 W | €429,000 excl. VAT | datasheet |
| Dell PowerEdge XE9680L (8x B200 DLC) | 8x NVIDIA Blackwell B200 SXM | 1,440 GB | 64.0 TB/s | 18 | 36 | 72 | 14,300 W | €494,000 excl. VAT | datasheet |
| NVIDIA DGX B200 (8x B200) | 8x NVIDIA Blackwell B200 SXM | 1,440 GB | 64.0 TB/s | 18 | 36 | 72 | 14,300 W | €527,000 excl. VAT | datasheet |
| NVIDIA DGX B300 (8x B300) | 8x NVIDIA Blackwell Ultra B300 SXM | 2,304 GB | 64.0 TB/s | 18 | 36 | 108 | 15,000 W | €736,000 excl. VAT | datasheet |
| NVIDIA GB200 NVL72 (complete rack, 72 GPUs) | 72x Blackwell B200 GPU | 13,392 GB | 576.0 TB/s | 180 | 360 | 720 | 132,000 W | €4,225,000 excl. VAT | datasheet |
| NVIDIA GB300 NVL72 (complete rack, 72 Blackwell Ultra GPUs) | 72x NVIDIA Blackwell Ultra B300 (288 GB HBM3e) | 20,736 GB | 576.0 TB/s | 180 | 360 | 1,080 | 135,000 W | €4,745,000 excl. VAT | datasheet |
| Lenovo ThinkSystem SR675 V3 (4x H100 SXM5, withdrawn configuration) | 4x NVIDIA H100 SXM5 | 320 GB | 13.4 TB/s | 3.96 | 7.92 | none | 3,600 W | price not surveyed | datasheet |
| HPE Cray EX255a (El Capitan) | 4x AMD Instinct MI300A per node (2 nodes and 8 APUs per blade) | 512 GB | 21.2 TB/s | 3.92 | 7.84 | none | 3,000 W | price not surveyed | datasheet |
| Apple Mac Studio M5 Ultra (512 GB unified, 1,200 GB/s) | Apple M5 Ultra, 80-core GPU with neural accelerators | 512 GB | 1.20 TB/s | 0.10 | none | none | 385 W | price not surveyed | datasheet |
| Apple Mac Studio M3 Ultra (256 GB unified, 2025) | Apple M3 Ultra, 80-core GPU | 256 GB | 0.82 TB/s | 0.03 | none | none | 270 W | price not surveyed | datasheet |
| AMD Helios (complete rack, 72 Instinct MI455X GPUs) | 72x AMD Instinct MI455X (432 GB HBM4) | 31,104 GB | 1,677.6 TB/s | 720 | 1,440 | 2,880 | n/a | price not surveyed | datasheet |
| Google TPU7x Ironwood pod (64-chip slice) | 64x Google TPU7x Ironwood chips | 13,194 GB | 472.3 TB/s | 148 | 295 | none | n/a | price not surveyed | datasheet |
| 2x NVIDIA H200 NVL (PCIe pair with NVLink bridge) | 2x NVIDIA H200 NVL PCIe (141 GB HBM3e each) | 282 GB | 9.60 TB/s | 1.67 | 3.34 | none | 2,000 W | price not surveyed | datasheet |
| Dell PowerEdge XE8640 (4x H100 SXM5) | 4x NVIDIA H100 SXM5 80 GB (700 W), linked by NVLink | 320 GB | 13.4 TB/s | 3.96 | 7.92 | none | 3,600 W | price not surveyed | datasheet |
| Dell PowerEdge XE9640 (4x H100 SXM5) | 4x NVIDIA H100 SXM5 80 GB (700 W), linked by NVLink | 320 GB | 13.4 TB/s | 3.96 | 7.92 | none | 3,600 W | price not surveyed | datasheet |
Open data
The calculator data is published as JSON: every machine with its memory, bandwidth, dense compute, power and the price used for the cost, and every model with its architecture. The file describes each field and its unit, and links to the vendor datasheets.
Download the dataset (JSON) · version 3.1, 12 October 2026
Which server for which model?
For each model, the cheapest machine in the catalogue that holds the weights and leaves at least a quarter of its memory for the cache, runtime reserve included. The cache depends on context and users: set them below. Withdrawn machines and machines without a surveyed price are left out.
| Model | FP8 weights | FP8 machine | NVFP4 weights | NVFP4 machine |
|---|---|---|---|---|
| Kimi K3 | 2,782 GB | NVIDIA GB200 NVL72 (complete rack, 72 GPUs) €4,225,000 excl. VAT · weights 21 % · 51 tok/s | 1,610 GB (published checkpoint) | NVIDIA DGX B300 (8x B300) €736,000 excl. VAT · weights 71 % · 49 tok/s |
| DeepSeek V4 Pro 0813 | 1,575 GB | 8x AMD Instinct MI355X OAM Server (CDNA 4) €380,000 excl. VAT · weights 70 % · 81 tok/s | 941 GB (published checkpoint) | Dell PowerEdge XE9680L (8x B200 DLC) €494,000 excl. VAT · weights 67 % · 79 tok/s |
| DeepSeek V4.1 Flash 1017 | 756 GB | Supermicro AS-8125GS-TNMR2 (8x MI300X) €277,000 excl. VAT · weights 51 % · 126 tok/s | 527 GB (published checkpoint) | Server 8x NVIDIA RTX PRO 6000 Blackwell Server Edition (768 GB) €192,000 excl. VAT · weights 72 % · 85 tok/s |
| GLM 5.3 | 756 GB (published checkpoint) | Supermicro AS-8125GS-TNMR2 (8x MI300X) €277,000 excl. VAT · weights 51 % · 63 tok/s | 464 GB (published checkpoint) | Server 8x NVIDIA RTX PRO 6000 Blackwell Server Edition (768 GB) €192,000 excl. VAT · weights 63 % · 41 tok/s |
| GLM 5.3 Flash | 328 GB (published checkpoint) | NVIDIA DGX A100 640 GB (8x A100 80 GB) €150,000 excl. VAT · weights 54 % · 101 tok/s | 204 GB (published checkpoint) | Server 8x NVIDIA RTX PRO 5000 Blackwell (384 GB) €96,000 excl. VAT · weights 57 % · 73 tok/s |
| Mistral Large 4 | 1,050 GB | Supermicro AS-8125GS-TNMR2 (8x MI300X) €277,000 excl. VAT · weights 70 % · cache not published: verdict limited to the weights | 591 GB | Dell PowerEdge XE9680L (8x B200 DLC) €494,000 excl. VAT · weights 42 % · cache not published: verdict limited to the weights |
| MiniMax M3 | 429 GB | NVIDIA DGX A100 640 GB (8x A100 80 GB) €150,000 excl. VAT · weights 70 % · 74 tok/s | 250 GB (published checkpoint) | Server 8x NVIDIA RTX PRO 5000 Blackwell (384 GB) €96,000 excl. VAT · weights 70 % · 54 tok/s |
| Qwen 3.5 | 398 GB | NVIDIA DGX A100 640 GB (8x A100 80 GB) €150,000 excl. VAT · weights 65 % · 79 tok/s | 251 GB (published checkpoint) | Server 8x NVIDIA RTX PRO 5000 Blackwell (384 GB) €96,000 excl. VAT · weights 70 % · 58 tok/s |
| Qwen 3.8 Flash Next | 179 GB | Dell PowerEdge R750xa (4x A100 PCIe) €59,000 excl. VAT · weights 59 % · 79 tok/s | 133 GB (published checkpoint) | Server 4x NVIDIA RTX PRO 5000 Blackwell (192 GB) €50,000 excl. VAT · weights 74 % · 73 tok/s |
| DeepSeek-V3 | 673 GB | Supermicro AS-8125GS-TNMR2 (8x MI300X) €277,000 excl. VAT · weights 45 % · 79 tok/s | 424 GB (published checkpoint) | Server 8x NVIDIA RTX PRO 6000 Blackwell Server Edition (768 GB) €192,000 excl. VAT · weights 57 % · 50 tok/s |
| DeepSeek Coder V2 | 237 GB | Server 8x NVIDIA RTX PRO 5000 Blackwell (384 GB) €96,000 excl. VAT · weights 66 % · 57 tok/s | 134 GB | Server 4x NVIDIA RTX PRO 5000 Blackwell (192 GB) €50,000 excl. VAT · weights 75 % · 55 tok/s |
| Llama 3.1 405B | 410 GB | NVIDIA DGX A100 640 GB (8x A100 80 GB) €150,000 excl. VAT · weights 67 % · 22 tok/s | 234 GB (published checkpoint) | Server 8x NVIDIA RTX PRO 5000 Blackwell (384 GB) €96,000 excl. VAT · weights 65 % · 23 tok/s |
| Nemotron-4 340B | 350 GB | NVIDIA DGX A100 640 GB (8x A100 80 GB) €150,000 excl. VAT · weights 57 % · 26 tok/s | 205 GB | Server 8x NVIDIA RTX PRO 5000 Blackwell (384 GB) €96,000 excl. VAT · weights 57 % · 27 tok/s |
| Mistral Large 2 | 123 GB | Apple Mac Studio M5 Ultra (256 GB unified, 1,200 GB/s) €10,358 excl. VAT · weights 60 % · 7.7 tok/s | 70.1 GB | Lenovo ThinkStation PGX GB10 €6,500 excl. VAT · weights 61 % · 2.4 tok/s |
| Llama 3.3 70B | 72.7 GB | Lenovo ThinkStation PGX GB10 €6,500 excl. VAT · weights 63 % · 2.4 tok/s | 42.7 GB (published checkpoint) | Lenovo ThinkStation PGX GB10 €6,500 excl. VAT · weights 37 % · 4.1 tok/s |
| Qwen 2.5 72B | 75.2 GB | Lenovo ThinkStation PGX GB10 €6,500 excl. VAT · weights 65 % · 2.3 tok/s | 44.5 GB | Lenovo ThinkStation PGX GB10 €6,500 excl. VAT · weights 38 % · 3.9 tok/s |
| Qwen 2.5 Coder 32B | 34.3 GB | AI Mini-PC AMD Ryzen AI Max+ 395 (Strix Halo 128 GB) €3,333 excl. VAT · weights 37 % · 4.6 tok/s | 20.7 GB | Lenovo ThinkStation PGX GB10 €6,500 excl. VAT · weights 18 % · 8.4 tok/s |
| Qwen 3.8 27B | 30.9 GB (published checkpoint) | AI Mini-PC AMD Ryzen AI Max+ 395 (Strix Halo 128 GB) €3,333 excl. VAT · weights 34 % · 5.7 tok/s | 21.9 GB (published checkpoint) | Lenovo ThinkStation PGX GB10 €6,500 excl. VAT · weights 19 % · 8.9 tok/s |
| Devstral Small 24B | 24.9 GB | AI Mini-PC AMD Ryzen AI Max+ 395 (Strix Halo 128 GB) €3,333 excl. VAT · weights 27 % · 6.5 tok/s | 15.2 GB | Mobile workstation RTX PRO 5000 Blackwell (24 GB) €5,208 excl. VAT · weights 72 % · 37 tok/s |
| Llama 3.1 8B | 9.1 GB | AI Mini-PC AMD Ryzen AI Max+ 395 (Strix Halo 128 GB) €3,333 excl. VAT · weights 10 % · 18 tok/s | 6.0 GB (published checkpoint) | Mobile workstation RTX PRO 5000 Blackwell (24 GB) €5,208 excl. VAT · weights 29 % · 95 tok/s |
Frequently asked questions
How much memory does it take to serve an LLM?
A model's weights take its parameters times the bytes of the format: two in BF16, one in FP8 and about 0.56 in NVFP4. The key-value cache comes on top and grows with the context and with the number of concurrent users.
How do I calculate the GPU memory of an LLM?
Add the model weights, the key-value cache and the state of recurrent layers. Per token, the cache is two times layers, KV heads, head dimension and bytes, multiplied by the context and the number of users. Then compare this total, GPU by GPU, with the usable memory of the machine.
Is the parameter count enough to compute the weights?
Almost. The calculator first takes the size of the published repository, including scales and layers kept in BF16, for example 892.7 GB for DeepSeek V4 Pro 0813. Otherwise it multiplies the parameters by the bytes of the format, then each GPU reserves 8% of its memory and 2.5 GiB at runtime, as vLLM does.
Why do the sizing sheets not count the key-value cache?
Because it depends on two operating choices the sheet does not know: the served context and the number of concurrent requests. The sheets judge the weights and keep a quarter of the memory free. This calculator counts the cache with your values and gives the maximum context per user.
Where do I find the cache values for a model missing from the list?
In the repository's config.json, which gives the number of layers, KV heads and the head dimension for per-head attention, or the latent rank and RoPE dimension for latent attention, while recurrent layers keep a fixed state per request.
How much GPU memory does QLoRA fine-tuning need?
The base weights stay frozen in NF4, about 0.52 byte per parameter, except embeddings and the output head kept in BF16. With TRL defaults, adapters, gradients and AdamW add 8 bytes per trained parameter. Activations grow with the sequence length.
What is an LLM sizer?
An LLM sizer is a sizing tool: it computes the memory, speed and number of users a machine can serve for a given model. This one compares twenty open models on more than sixty servers, from a mini-PC to an NVIDIA GB300 NVL72 or AMD Helios rack, and derives the cost per million tokens.
Which server do I need for Kimi K3?
In FP8, the weights of Kimi K3 take 2,782 GB. The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the NVIDIA GB200 NVL72 (complete rack, 72 GPUs), sold for €4,225,000 excl. VAT, with a calculated speed of 51 tokens per second per concurrent user.
In NVFP4, on the Blackwell hardware that runs it natively, the weights of Kimi K3 take 1,610 GB (checkpoint nvidia/Kimi-K3-NVFP4). The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the NVIDIA DGX B300 (8x B300), sold for €736,000 excl. VAT, with a calculated speed of 49 tokens per second per concurrent user.
Which server do I need for DeepSeek V4 Pro 0813?
In FP8, the weights of DeepSeek V4 Pro 0813 take 1,575 GB. The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the 8x AMD Instinct MI355X OAM Server (CDNA 4), sold for €380,000 excl. VAT, with a calculated speed of 81 tokens per second per concurrent user.
In NVFP4, on the Blackwell hardware that runs it natively, the weights of DeepSeek V4 Pro 0813 take 941 GB (checkpoint nvidia/DeepSeek-V4-Pro-0813-NVFP4). The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the Dell PowerEdge XE9680L (8x B200 DLC), sold for €494,000 excl. VAT, with a calculated speed of 79 tokens per second per concurrent user.
Which server do I need for DeepSeek V4.1 Flash 1017?
In FP8, the weights of DeepSeek V4.1 Flash 1017 take 756 GB. The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the Supermicro AS-8125GS-TNMR2 (8x MI300X), sold for €277,000 excl. VAT, with a calculated speed of 126 tokens per second per concurrent user.
In NVFP4, on the Blackwell hardware that runs it natively, the weights of DeepSeek V4.1 Flash 1017 take 527 GB (checkpoint nvidia/DeepSeek-V4.1-Flash-NVFP4). The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the Server 8x NVIDIA RTX PRO 6000 Blackwell Server Edition (768 GB), sold for €192,000 excl. VAT, with a calculated speed of 85 tokens per second per concurrent user.
Which server do I need for GLM 5.3?
In FP8, the weights of GLM 5.3 take 756 GB (checkpoint zai-org/GLM-5.3). The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the Supermicro AS-8125GS-TNMR2 (8x MI300X), sold for €277,000 excl. VAT, with a calculated speed of 63 tokens per second per concurrent user.
In NVFP4, on the Blackwell hardware that runs it natively, the weights of GLM 5.3 take 464 GB (checkpoint nvidia/GLM-5.3-NVFP4). The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the Server 8x NVIDIA RTX PRO 6000 Blackwell Server Edition (768 GB), sold for €192,000 excl. VAT, with a calculated speed of 41 tokens per second per concurrent user.
Which server do I need for GLM 5.3 Flash?
In FP8, the weights of GLM 5.3 Flash take 328 GB (checkpoint zai-org/GLM-5.3-Flash). The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the NVIDIA DGX A100 640 GB (8x A100 80 GB), sold for €150,000 excl. VAT, with a calculated speed of 101 tokens per second per concurrent user.
In NVFP4, on the Blackwell hardware that runs it natively, the weights of GLM 5.3 Flash take 204 GB (checkpoint nvidia/GLM-5.3-Flash-NVFP4). The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the Server 8x NVIDIA RTX PRO 5000 Blackwell (384 GB), sold for €96,000 excl. VAT, with a calculated speed of 73 tokens per second per concurrent user.
Which server do I need for Mistral Large 4?
In FP8, the weights of Mistral Large 4 take 1,050 GB. The cache architecture is not published, so the calculator only states that the Supermicro AS-8125GS-TNMR2 (8x MI300X), sold for €277,000 excl. VAT, holds these weights with a quarter of its memory free, without a verdict on context or speed.
In NVFP4, on the Blackwell hardware that runs it natively, the weights of Mistral Large 4 take 591 GB. The cache architecture is not published, so the calculator only states that the Dell PowerEdge XE9680L (8x B200 DLC), sold for €494,000 excl. VAT, holds these weights with a quarter of its memory free, without a verdict on context or speed.
Which server do I need for MiniMax M3?
In FP8, the weights of MiniMax M3 take 429 GB. The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the NVIDIA DGX A100 640 GB (8x A100 80 GB), sold for €150,000 excl. VAT, with a calculated speed of 74 tokens per second per concurrent user.
In NVFP4, on the Blackwell hardware that runs it natively, the weights of MiniMax M3 take 250 GB (checkpoint nvidia/MiniMax-M3-NVFP4). The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the Server 8x NVIDIA RTX PRO 5000 Blackwell (384 GB), sold for €96,000 excl. VAT, with a calculated speed of 54 tokens per second per concurrent user.
Which server do I need for Qwen 3.5?
In FP8, the weights of Qwen 3.5 take 398 GB. The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the NVIDIA DGX A100 640 GB (8x A100 80 GB), sold for €150,000 excl. VAT, with a calculated speed of 79 tokens per second per concurrent user.
In NVFP4, on the Blackwell hardware that runs it natively, the weights of Qwen 3.5 take 251 GB (checkpoint nvidia/Qwen3.5-397B-A17B-NVFP4). The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the Server 8x NVIDIA RTX PRO 5000 Blackwell (384 GB), sold for €96,000 excl. VAT, with a calculated speed of 58 tokens per second per concurrent user.
Which server do I need for Qwen 3.8 Flash Next?
In FP8, the weights of Qwen 3.8 Flash Next take 179 GB. The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the Dell PowerEdge R750xa (4x A100 PCIe), sold for €59,000 excl. VAT, with a calculated speed of 79 tokens per second per concurrent user.
In NVFP4, on the Blackwell hardware that runs it natively, the weights of Qwen 3.8 Flash Next take 133 GB (checkpoint nvidia/Qwen3.8-Flash-Next-NVFP4). The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the Server 4x NVIDIA RTX PRO 5000 Blackwell (192 GB), sold for €50,000 excl. VAT, with a calculated speed of 73 tokens per second per concurrent user.
Which server do I need for DeepSeek-V3?
In FP8, the weights of DeepSeek-V3 take 673 GB. The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the Supermicro AS-8125GS-TNMR2 (8x MI300X), sold for €277,000 excl. VAT, with a calculated speed of 79 tokens per second per concurrent user.
In NVFP4, on the Blackwell hardware that runs it natively, the weights of DeepSeek-V3 take 424 GB (checkpoint nvidia/DeepSeek-V3-0324-NVFP4). The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the Server 8x NVIDIA RTX PRO 6000 Blackwell Server Edition (768 GB), sold for €192,000 excl. VAT, with a calculated speed of 50 tokens per second per concurrent user.
Which server do I need for DeepSeek Coder V2?
In FP8, the weights of DeepSeek Coder V2 take 237 GB. The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the Server 8x NVIDIA RTX PRO 5000 Blackwell (384 GB), sold for €96,000 excl. VAT, with a calculated speed of 57 tokens per second per concurrent user.
In NVFP4, on the Blackwell hardware that runs it natively, the weights of DeepSeek Coder V2 take 134 GB. The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the Server 4x NVIDIA RTX PRO 5000 Blackwell (192 GB), sold for €50,000 excl. VAT, with a calculated speed of 55 tokens per second per concurrent user.
Which server do I need for Llama 3.1 405B?
In FP8, the weights of Llama 3.1 405B take 410 GB. The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the NVIDIA DGX A100 640 GB (8x A100 80 GB), sold for €150,000 excl. VAT, with a calculated speed of 22 tokens per second per concurrent user.
In NVFP4, on the Blackwell hardware that runs it natively, the weights of Llama 3.1 405B take 234 GB (checkpoint nvidia/Llama-3.1-405B-Instruct-NVFP4). The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the Server 8x NVIDIA RTX PRO 5000 Blackwell (384 GB), sold for €96,000 excl. VAT, with a calculated speed of 23 tokens per second per concurrent user.
Which server do I need for Nemotron-4 340B?
In FP8, the weights of Nemotron-4 340B take 350 GB. The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the NVIDIA DGX A100 640 GB (8x A100 80 GB), sold for €150,000 excl. VAT, with a calculated speed of 26 tokens per second per concurrent user.
In NVFP4, on the Blackwell hardware that runs it natively, the weights of Nemotron-4 340B take 205 GB. The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the Server 8x NVIDIA RTX PRO 5000 Blackwell (384 GB), sold for €96,000 excl. VAT, with a calculated speed of 27 tokens per second per concurrent user.
Which server do I need for Mistral Large 2?
In FP8, the weights of Mistral Large 2 take 123 GB. The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the Apple Mac Studio M5 Ultra (256 GB unified, 1,200 GB/s), sold for €10,358 excl. VAT, with a calculated speed of 7.7 tokens per second per concurrent user.
In NVFP4, on the Blackwell hardware that runs it natively, the weights of Mistral Large 2 take 70.1 GB. The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the Lenovo ThinkStation PGX GB10, sold for €6,500 excl. VAT, with a calculated speed of 2.4 tokens per second per concurrent user.
Which server do I need for Llama 3.3 70B?
In FP8, the weights of Llama 3.3 70B take 72.7 GB. The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the Lenovo ThinkStation PGX GB10, sold for €6,500 excl. VAT, with a calculated speed of 2.4 tokens per second per concurrent user.
In NVFP4, on the Blackwell hardware that runs it natively, the weights of Llama 3.3 70B take 42.7 GB (checkpoint nvidia/Llama-3.3-70B-Instruct-NVFP4). The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the Lenovo ThinkStation PGX GB10, sold for €6,500 excl. VAT, with a calculated speed of 4.1 tokens per second per concurrent user.
Which server do I need for Qwen 2.5 72B?
In FP8, the weights of Qwen 2.5 72B take 75.2 GB. The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the Lenovo ThinkStation PGX GB10, sold for €6,500 excl. VAT, with a calculated speed of 2.3 tokens per second per concurrent user.
In NVFP4, on the Blackwell hardware that runs it natively, the weights of Qwen 2.5 72B take 44.5 GB. The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the Lenovo ThinkStation PGX GB10, sold for €6,500 excl. VAT, with a calculated speed of 3.9 tokens per second per concurrent user.
Which server do I need for Qwen 2.5 Coder 32B?
In FP8, the weights of Qwen 2.5 Coder 32B take 34.3 GB. The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the AI Mini-PC AMD Ryzen AI Max+ 395 (Strix Halo 128 GB), sold for €3,333 excl. VAT, with a calculated speed of 4.6 tokens per second per concurrent user.
In NVFP4, on the Blackwell hardware that runs it natively, the weights of Qwen 2.5 Coder 32B take 20.7 GB. The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the Lenovo ThinkStation PGX GB10, sold for €6,500 excl. VAT, with a calculated speed of 8.4 tokens per second per concurrent user.
Which server do I need for Qwen 3.8 27B?
In FP8, the weights of Qwen 3.8 27B take 30.9 GB (checkpoint Qwen/Qwen3.8-27B-FP8). The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the AI Mini-PC AMD Ryzen AI Max+ 395 (Strix Halo 128 GB), sold for €3,333 excl. VAT, with a calculated speed of 5.7 tokens per second per concurrent user.
In NVFP4, on the Blackwell hardware that runs it natively, the weights of Qwen 3.8 27B take 21.9 GB (checkpoint nvidia/Qwen3.8-27B-NVFP4). The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the Lenovo ThinkStation PGX GB10, sold for €6,500 excl. VAT, with a calculated speed of 8.9 tokens per second per concurrent user.
Which server do I need for Devstral Small 24B?
In FP8, the weights of Devstral Small 24B take 24.9 GB. The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the AI Mini-PC AMD Ryzen AI Max+ 395 (Strix Halo 128 GB), sold for €3,333 excl. VAT, with a calculated speed of 6.5 tokens per second per concurrent user.
In NVFP4, on the Blackwell hardware that runs it natively, the weights of Devstral Small 24B take 15.2 GB. The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the Mobile workstation RTX PRO 5000 Blackwell (24 GB), sold for €5,208 excl. VAT, with a calculated speed of 37 tokens per second per concurrent user.
Which server do I need for Llama 3.1 8B?
In FP8, the weights of Llama 3.1 8B take 9.1 GB. The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the AI Mini-PC AMD Ryzen AI Max+ 395 (Strix Halo 128 GB), sold for €3,333 excl. VAT, with a calculated speed of 18 tokens per second per concurrent user.
In NVFP4, on the Blackwell hardware that runs it natively, the weights of Llama 3.1 8B take 6.0 GB (checkpoint nvidia/Llama-3.1-8B-Instruct-NVFP4). The cheapest machine in the catalogue that keeps a quarter of its memory for the key-value cache is the Mobile workstation RTX PRO 5000 Blackwell (24 GB), sold for €5,208 excl. VAT, with a calculated speed of 95 tokens per second per concurrent user.
Further reading
- Sizing sheets by model and machine
- Prefill, decode and key-value cache: what the hardware measures
- Qwen 3.8 27B: 262,144 tokens of context on one machine
- Mistral Large 4: verified facts and hardware
- Cost comparator: per-token API or on-premises server
- GPU server rental price comparator (Nebius, NVIDIA Brev, all brands)
- Sizing the LiteLLM gateway in front of this service