GLM 5.3 deep on-premise: which hardware, at what speed?

By QDNA · Published on 18 August 2026 · Updated on 11 October 2026

In-depth study of the open-weight model from Zhipu AI announced on August 14, 2026 with weights published on Hugging Face on August 28, 2026, and a hardware sizing guide for running it locally.

Updated on 11 October 2026: throughput figures withdrawn. The values published since August 18 for the DGX Spark x2 (~28 tok/s) and the DGX Station (~45 tok/s) referred to the 756 GB FP8 weights, which fit in neither machine. The hardware table now shows the format that actually fits, and costs per million tokens are back to "[TO BE MEASURED]".

Updated on 8 October 2026: reference prices of the DGX Spark (€7,000 excl. VAT) and the DGX Station GB300 (€150,000 excl. VAT) raised. Cost per million tokens recomputed with the site formula (€0.94 instead of €0.06 on DGX Spark x2); the break-even point and the year-two claim are removed.

Updated on 26 September 2026: official nvidia/GLM-5.3-NVFP4 conversion published on September 14, 2026 (464 GB), Artificial Analysis index v4.3.2 (GLM 5.3 at 45), vLLM 0.30.0.

Updated on September 2, 2026: every figure re-read against primary sources. Release dates corrected (GLM 5: February 2026; GLM 5.3: announced August 14, weights August 28, 2026), KV cache recomputed from config.json (44 KiB per token in FP8, not 64), DGX Station (748 GB coherent) and B200 x8 server (1,440 GB) memory aligned with NVIDIA sheets, NVFP4 size taken from NVIDIA's GLM 5.2 conversion (465 GB) instead of a calculation, unmeasured throughputs replaced by "[TO BE MEASURED]", unsourced claims removed (30 users, 18 queries per second, p95, 4,000-word threshold).

Updated on September 1, 2026: score table and comparison corrected against the Hugging Face model card of GLM 5.3 and the repositories of the compared models. The August 18 version cited a model that does not exist ("Qwen 3.8 Max 235B"), wrong licences (Llama 3.1 "CC-BY-NC") and an unsourced "Intelligence Index" column; the weights published by Z.ai are FP8 (756 GB), not BF16.

Methodology

The methodology of this study cross-references three sources, the vendor's official announcements, the scores Z.ai publishes on its own harness and QDNA's sizing rules (weight footprint, KV cache), without showing any third-party index as a column:

1. Official announcements from Zhipu AI and THUDM, supplemented by the model's Hugging Face card and the NVFP4 conversion published by NVIDIA.
2. The scores published by Z.ai on the model's Hugging Face card, measured by Z.ai on its own harness for every compared model (comparable with each other, but produced by a competitor). No third-party index is shown as a column: the Artificial Analysis Intelligence Index (v4.1.1, consulted September 1, 2026) does not cover every model cited.
3. No throughput figure without a reproducible measurement on QDNA hardware or a named third-party source: unmeasured configurations stay without a value.

Throughput numbers that we have not verified on our own hardware are marked "[TO BE MEASURED]" in the tables below. Benchmark numbers are quoted as sourced with no extrapolation; the throughput figures announced at first publication (August 18, 2026) were withdrawn on October 11, 2026, while scores, comparison and every configuration value were re-read in the Hugging Face repositories and on vendor sheets on September 2, 2026.

Official sources

• GLM 5: from agentic to engineering, Z.ai (February 2026; technical report arXiv 2602.15763)
• zai-org/GLM-5.3, official model card and config.json
• GLM 5.3 Flash model card, Z.ai on Hugging Face
• zai-org/GLM-5.2 (same base model, MIT licence, BF16 weights)
• Z.ai GLM-5 README on GitHub (744B-A40B, "GLM-5.3 uses the same base model as GLM-5.2")
• nvidia/GLM-5.3-NVFP4 (official conversion of September 14, 2026, 464 GB) and nvidia/GLM-5.2-NVFP4 (same base)
• Artificial Analysis, open-weight models (Intelligence Index v4.3.2, consulted September 26, 2026: GLM 5.3 at 45, second open model behind MiMo-V2.6-Pro at 46; not shown as a column)

Model architecture

GLM 5.3 keeps the architectural family introduced by GLM 5, a mixture of experts with sparse attention, and the structure published by THUDM reports the following:

• 744 billion total parameters.
• ~40 billion active parameters per token (256 routed experts, one shared expert).
• 78 layers, attention heads in sparse DSA (DeepSeek Sparse Attention), an indexer that keeps the 2,048 best-scored positions per token (index_topk in config.json) and MLA latent attention (kv_lora_rank 512, 64-wide RoPE head).
• Extended GLM-2 tokenizer, 154,880-entry vocabulary.
• 1,048,576-token context window, identical to GLM 5.2.
• The sizing tables on this page assume a 200,000-token KV cache budget, which is not the model's window: serving the full million takes far more memory.

The mixture of experts activates about 5% of weights at each step (40 of 744 billion), which bounds throughput; total size bounds memory: the 756 GB of published FP8 weights only fit as-is on an eight-GPU server (B200 x8, 1,440 GB), and a single node such as a DGX Station (748 GB) requires 4-bit requantization (464 GB read on September 26, 2026 on NVIDIA's official NVFP4 conversion of GLM 5.3). The marginal cost per token is still higher than an equivalent dense model, because memory bandwidth stays dominated by total weight.

Official benchmarks

The table below reproduces the scores published by Z.ai on the GLM 5.3 Hugging Face model card, re-read on September 1, 2026.

Z.ai measured every model on its own harness (Claude Code 2.1.207 for Terminal Bench 3.0, parameters published in the card's footnotes): the columns are comparable with each other, but these are the figures of a competitor of the compared models, not their vendors' nor a neutral third party's. The card publishes neither MMLU-Pro, nor GPQA, nor AIME, nor SWE-Bench Verified: we no longer report them.

Quick read. On Z.ai's harness, GLM 5.3 improves sharply on GLM 5.2 at equal weight (Terminal Bench 2.1 from 81.0 to 88.2, DeepSWE from 46.2 to 66.9, CyberGym from 77.2 to 84.5): the gain comes from post-training, the base model being the same. It sits level with Kimi K3 on agentic tasks and ahead of it on CyberGym and HLE with tools, and it still trails Claude Opus 4.8 on NL2Repo and Toolathlon. The vendors' own scores (DeepSeek, Moonshot, Alibaba, Anthropic) are measured under other conditions and cannot be compared with this table.

Available quantizations

Z.ai publishes GLM 5.3 in FP8: the repository's config.json declares an fp8 quantization (128 × 128 blocks) and the safetensors files weigh 756 GB, whereas GLM 5.2 was published in BF16 (1,507 GB). The other formats are calculations or community conversions.

DGX Spark and DGX Station are Blackwell parts and expose FP4 (NVIDIA advertises 1 petaFLOP FP4 with sparsity on the GB10): the choice between the published FP8 and an NVFP4 conversion comes down to available memory and to the conversion one is willing to install, not to a hardware limit.

Hardware: which GPU for which target?

The table below crosses five hardware configurations we regularly size at QDNA, with the GLM 5.3 format that fits in memory and the status of throughput measurement, where "practical context" means the maximum context window without degrading single-user throughput by more than 10%.

The throughput figures published until October 11, 2026 for the DGX Spark x2 (~28 tok/s) and the DGX Station (~45 tok/s) were withdrawn: they were given in FP8, a 756 GB format that fits in neither machine. No configuration is measured to date, and no named third-party source publishes GLM 5.3 throughput on the DGX Station, the B200 x8 or the GB300 NVL72: the cells stay "[TO BE MEASURED]" until a reproducible measurement is published in the measurements section.

Operating cost

The hourly cost follows the site's common formula: purchase price excl. VAT divided by 26,280 hours (three years), plus power in kW billed at €0.1624 per kWh. It comes to €5.97 for the DGX Station GB300 (€150,000, 1,600 W), the only single-node machine in the table that holds the model.

The cost per million tokens divides the hourly cost by the aggregate throughput times 3,600 seconds times the load, 30% being the site's median assumption. With no measured throughput, it stays "[TO BE MEASURED]" for every configuration in the table.

The break-even point does not depend on throughput. Against the DeepSeek V4 Flash API at $0.18/M, or €0.16 at the ECB rate of $1.1403, a DGX Station costing €4,356 per month only breaks even at 27.6 billion tokens per month, about 10,600 tokens per second around the clock. The case for running locally is sovereignty, not the price per token.

Comparison with open-weight competitors

Values re-read on September 1, 2026 in the config.json, README and licence of each Hugging Face repository; the detail is in our open-weight models comparison. No third-party index or cost per token is shown: the Artificial Analysis index does not cover all of these models, and costs depend on the machine, not on the model.

GLM 5.3 becomes the reference for frontal use in self-hosting: level with Kimi K3 on Z.ai's harness for half the published weight, a native one-million-token context of which the 200K budget retained here covers most uses, and a GLM-5.3 licence whose only condition (a Z.ai security review for a model-as-a-service operator above 10 billion dollars of revenue over twelve months) does not concern an enterprise deployment. Kimi K3 stays at the same level on paper, but its 1,561 GB of published weights exceed every single-node machine in the catalogue.

Comparison with closed models

On Z.ai's harness, GLM 5.3 beats Claude Opus 4.8 on Terminal Bench 2.1 and 3.0, DeepSWE, CyberGym and HLE with tools, and trails it on NL2Repo and Toolathlon Verified (table above).

The vendors of closed models publish no scores on this harness, and we show neither a third-party index nor API prices: they are read on each vendor's price list. GLM 5.3's advantage remains local deployment without any third-party API dependency.

Suitable use cases

The suitable use cases for GLM 5.3 are those where Z.ai's harness places it level with the best open models: code agents, long-context retrieval-augmented generation and tool-augmented generation.

• Code agents in production. On Z.ai's harness, GLM 5.3 reaches 88.2 on Terminal Bench 2.1 and 66.9 on DeepSWE, level with Kimi K3 (88.3 and 67.5) and ahead of Qwen3.8-Max (86.6 and 56.6).
• Long-context retrieval-augmented generation (RAG). 200K tokens cover a dossier of about 500 A4 pages (assumption: 300 words per page and 1.3 tokens per word, i.e. 390 tokens per page; 200,000 / 390 ≈ 510). The queries per second of a RAG chain depend on the index and the hardware: [TO BE MEASURED].
• Tool-augmented generation. Terminal Bench 2.1 at 88.2 (Z.ai harness) indicates solid shell mastery, Navigation API, structured JSON. Compatible with OpenCode, Hermes Agent, internal harnesses.
• Long-form writing in French. French quality is slightly behind English but the GLM-2 tokenizer extends multilingual coverage; no public benchmark quantifies that gap.

When not to pick GLM 5.3?

Do not pick GLM 5.3 when pure mathematical reasoning dominates, when a single DGX Spark is all you have, when latency matters more than quality, or when the context regularly exceeds 200K.

• If the primary task is pure mathematical reasoning, proprietary models remain the reference; their vendors publish no scores on Z.ai's harness, so we do not quantify the gap.
• If the organization is strictly limited to a single DGX Spark (128 GB), Qwen 3.8 27B dense in FP8 fits on a single node without offload.
• If latency dominates over quality, a model with fewer active parameters (GLM 5.3 Flash, 320 billion of which 18 active; Qwen3.8-Flash-Next, 125 billion of which 6 active) serves faster on the same hardware, since throughput is bounded by active parameters.
• If input context regularly exceeds 200K, the KV cache budget, not the model window, becomes the constraint: on the same machine, DeepSeek V4 Flash 0731 (167 GB published) leaves more memory to the cache than a requantized GLM 5.3 (465 GB).

Production deployment: checklist

The production deployment checklist follows the order of dependencies: pull the weights, pick the runtime, compute the KV cache, size the hardware, then measure before opening the service.

1. Pull the FP8 weights published by Z.ai on Hugging Face (756 GB), the ModelScope mirror, or an NVFP4 conversion.
2. Pick the runtime: vLLM (v0.30.0, latest release on GitHub as of September 26, 2026) for production, llama.cpp for debugging, SGLang for high-traffic agentic workloads.
3. Compute the KV cache based on the target window. GLM 5.3 stores per layer a 512-value MLA latent and a 64-wide RoPE key (config.json): (512 + 64) × 78 layers = 44,928 bytes, i.e. 44 KiB per token with an FP8 cache and 88 KiB in BF16, excluding the DSA indexer keys (128 bytes per layer).
4. Size hardware via the QDNA sheet of the chosen GPU (DGX Spark, DGX Station, H200, B200, B300, GB300 NVL72).
5. Serve via OpenAI-compatible endpoint, hook into LiteLLM, add the semantic router if multiple models coexist.
6. Measure throughput in single-user then load mode with Locust or k6, and set the p95 latency target from that measurement.
7. Connect monitoring: inter-token latency, NCCL RTT (if clustered), VRAM saturation, KV truncation rate.

To go further

To go further, the resources below cover the comparison of open-weight models, the independent benchmark, the DeepSeek comparison and the QDNA sheets for the model and the hardware it runs on.

• Open-weight models for local deployment 2026: full comparison
• Independent benchmark of open-weight models 2026
• Benchmark DeepSeek V4 Flash 0731 vs Claude Opus 5 vs GPT-5.5
• GLM model sheet on QDNA
• DGX Station GB300 hardware sheet
• DGX Spark hardware sheet
• GLM in FP8
• GLM in NVFP4