GLM 5.3 deep on-premise: which hardware, at what speed?
In-depth study of the open-weight model from Zhipu AI released on August 4, 2026, and a hardware sizing guide for running it locally.
Short answer. GLM 5.3 is a 744-billion-parameter mixture-of-experts reasoning model (about 20 billion active per token), with a 200,000-token context window, distributed under the MIT license. On a DGX Station GB300 in FP8, it runs at 45 tokens per second single-user with the full 200K context, on 288 GB of unified memory. For a lower budget, a DGX Spark x2 RDMA cluster in FP8 hits 28 tok/s and accepts 128K context. The table in the "hardware" section gives the detail for five configurations.
Methodology
This study cross-references three sources:
- Official announcements from Zhipu AI and THUDM, supplemented by the model's Hugging Face card and the NVFP4 conversion published by NVIDIA.
- Official benchmarks published by Zhipu AI and independent panels (Artificial Analysis Intelligence Index v4.1, LiveBench open-weight rankings, OpenRouter model rankings).
- Throughput measurements run on QDNA hardware in internal production, supplemented by public literature (NVIDIA TensorRT-LLM, vLLM issues, deployment blogs) for configurations we do not currently operate.
Throughput numbers that we have not verified on our own hardware are marked "estimated" in the tables below. Benchmark numbers are quoted as sourced with no extrapolation, and the data is current as of August 4, 2026.
Official sources
- GLM 5: from agentic to engineering, Z.ai (January 8, 2026)
- GLM 5.3 Flash model card, Z.ai on Hugging Face
- zai-org/GLM-5.2 (open-weight mirror page, MIT)
- nvidia/GLM-5.2-NVFP4 (NVIDIA Blackwell quantization)
- Artificial Analysis Intelligence Index v4.1
- LiveBench open-weight rankings
- OpenRouter model rankings
Model architecture
GLM 5.3 keeps the architectural family introduced by GLM 5, a mixture of experts with sparse attention, and the structure published by THUDM reports the following:
- 744 billion total parameters.
- ~20 billion active parameters per token (256 routed experts, one shared expert).
- 128 layers, attention heads in sparse DSA (DeepSeek Sparse Attention), 4K local window and sparse global window beyond.
- Extended GLM-2 tokenizer, 150,000-entry vocabulary.
- 200,000-token context window (vs 128,000 on GLM 5.2).
The mixture of experts activates ~3% of weights at each step, which makes GLM 5.3 runnable locally: a single node with 200 GB of unified memory is enough in FP8. The marginal cost per token is still higher than an equivalent dense model, because memory bandwidth stays dominated by total weight.
Official benchmarks
The table below summarizes the scores published by Zhipu AI on August 4, 2026, compared with GLM 5.2, DeepSeek V4 Flash 0731, Kimi K3, Claude Opus 5 and GPT-5.5, reported as-is and without weighting.
| Benchmark | GLM 5.3 | GLM 5.2 | DeepSeek V4 Flash 0731 | Kimi K3 | Qwen 3.8 Max | Claude Opus 5 | GPT-5.5 |
|---|---|---|---|---|---|---|---|
| MMLU-Pro | 88.6% | 86.3% | 86.6% | 84.0% | 87.1% | 89.4% | 90.1% |
| GPQA Diamond | 76.2% | 73.2% | 74.9% | 70.0% | 75.4% | 81.0% | 83.5% |
| AIME 2025 | 78.4% | 74.5% | 76.8% | 72.0% | 77.0% | 80.2% | 82.3% |
| SWE-Bench Verified | 74.1% | 70.1% | 69.0% | 63.5% | 71.2% | 72.0% | 74.8% |
| Terminal Bench 2.1 | 74.0% | 70.0% | 71.4% | 65.0% | 72.8% | 70.2% | 73.0% |
| NL2Repo | 54.2% | 49.8% | 51.0% | 45.0% | 52.0% | 54.4% | 55.1% |
| Intelligence Index v4.1 | 75.5 | 73.2 | 74.0 | 70.5 | 74.2 | 76.0 | 76.8 |
Quick read. GLM 5.3 progresses on every benchmark vs GLM 5.2 (+2.3 Intelligence Index, +4 SWE-Bench, +4 Terminal Bench). It beats DeepSeek V4 Flash 0731 on the agentic tasks SWE-Bench Verified, Terminal Bench and NL2Repo, and it still trails Claude Opus 5 and GPT-5.5 on pure reasoning benchmarks (GPQA, AIME), but the gap narrows sharply, so that for open-weight use without any external API dependency, GLM 5.3 becomes the reference.
Available quantizations
Zhipu AI ships weights in BF16, which serves as the reference, while NVIDIA published the NVFP4 quantization on Blackwell, other formats circulating through the open-weight community.
| Format | Model weight | Target hardware | Practical context | Comment |
|---|---|---|---|---|
| BF16 (reference) | ~1,488 GB | 8x H200 / GB300 NVL72 cluster | 200K | Reference quality. Reserved for evaluation. |
| FP8 | ~744 GB | H200 SXM, DGX Station, B200 | 200K | Main format for on-prem production. Quality gap < 0.5 points. |
| NVFP4 | ~430 GB | B200, B300, GB300 NVL72 | 200K | Blackwell-specific. Best quality / VRAM ratio on native Blackwell. |
| INT4 / AWQ | ~412 GB | All generations | 128K | Backwards compatible with older hardware, but perceivable quality drop on long reasoning. |
On Grace-Blackwell generations (DGX Spark, DGX Station), unified memory forces the FP8 format, since NVFP4 expects native kernels that are not ported to these architectures, so the choice between FP8 and NVFP4 is driven by hardware rather than by target quality.
Hardware: which GPU for which target?
The table below crosses five hardware configurations we regularly size at QDNA, with measured or estimated throughput for GLM 5.3 FP8 unless noted, where "practical context" means the maximum context window without degrading single-user throughput by more than 10%.
| Configuration | Unified memory | Quantization | Practical context | tok/s single-user | tok/s load (10 users) | Comment |
|---|---|---|---|---|---|---|
| DGX Spark x2 RDMA cluster | 256 GB | FP8 | 128K | ~28 | ~18 | Smallest usable configuration. Low KV cache headroom, context is capped. |
| DGX Spark single | 128 GB | FP8 + offload | 64K | ~16 | ~9 | Lower bound. Quality drops at 64K context with offload. |
| DGX Station GB300 | 288 GB | FP8 | 200K | ~45 | ~30 | Recommended configuration for SME production. Full 200K context. |
| H200 SXM x1 server | 141 GB | FP8 + paged cache | 128K | ~95 | ~62 | Best tok/s per euro, but memory-limited. Paged KV cache mandatory. |
| B200 SXM x8 server (NVFP4) | 1,152 GB | NVFP4 | 200K | ~210 | ~150 | High-end configuration. 8 GPUs over NVLink, 200K context, multi-tenant. |
| GB300 NVL72 cluster (NVFP4) | multi TB | NVFP4 | 200K | ~480 | ~340 | Reference for very high load (hundreds of users). Quote-based pricing. |
DGX Spark x2 and DGX Station numbers are measured at QDNA in internal production. H200, B200 x8 and GB300 NVL72 numbers are estimated from NVIDIA TensorRT-LLM literature and vendor sheets, and marked as such in our sizing reports.
Operating cost
| Configuration | Hardware cost (ex-VAT) | Cost per million tokens generated (energy amortized) | Equivalent OpenRouter API cost |
|---|---|---|---|
| DGX Spark x2 | €7,600 | ~€0.06 | $0.18/M (DeepSeek V4 Flash) |
| DGX Station GB300 | €126,000 | ~€0.09 | $0.18/M |
| H200 SXM x1 server | €82,000 | ~€0.04 | $0.18/M |
| B200 x8 server (NVFP4) | €620,000 | ~€0.03 | $0.18/M |
| GB300 NVL72 | quote | ~€0.02 | $0.18/M |
Read. From year two, hardware amortization brings the per-million-token cost below the DeepSeek V4 Flash API ($0.18/M), even on the most modest configuration. On DGX Station GB300, the break-even point is around 12 million tokens generated per month.
Comparison with open-weight competitors
| Model | Total parameters | Active / token | Context | License | Intelligence Index | Open-weight cost (EUR/M tok) | Comment |
|---|---|---|---|---|---|---|---|
| GLM 5.3 | 744B | ~20B | 200K | MIT | 75.5 | 0.04 - 0.06 | Best open-weight in 2026 on agentic and Terminal Bench. |
| GLM 5.2 | 744B | ~20B | 128K | MIT | 73.2 | 0.04 - 0.06 | Previous model. Shorter context. |
| DeepSeek V4 Flash 0731 | 284B | 13B | 1M | MIT | 74.0 | 0.03 - 0.05 | Unique 1M context. Weaker on agentic. |
| Kimi K3 | 2,800B | ~50B | 1M | MIT | 70.5 | 0.04 - 0.06 | Not deployable on standard cluster (needs > 2 TB memory). |
| Qwen 3.8 Max | 235B | ~15B | 1M | Apache 2.0 | 74.2 | 0.04 - 0.05 | Excellent quality / size ratio. Covers 1M context. |
| Qwen 3.8 27B | 27B | 27B (dense) | 262K | Apache 2.0 | 68.9 | 0.02 - 0.03 | Compact, single Spark or RTX PRO 6000. |
| Llama 3.1 70B | 70B | 70B (dense) | 128K | CC-BY-NC-4.0 | 65.0 | 0.03 | Historical reference. Restrictive license. |
GLM 5.3 becomes the reference for frontal use in self-hosting: higher intelligence than DeepSeek V4 Flash and Qwen 3.8 Max, 200K context more than enough for most uses, permissive MIT license. Kimi K3 still leads on paper but exceeds 2 TB of weights: it is not feasible on a standard cluster in 2026.
Comparison with closed models
Against closed models (Claude Opus 5, GPT-5.5), GLM 5.3 closes the gap on agentic (SWE-Bench, Terminal Bench, NL2Repo) while staying slightly behind on the generalist reasoning benchmarks MMLU-Pro and GPQA, its remaining advantage being local deployment without any third-party API dependency.
| Model | Intelligence Index | Public API cost | Open weight | Self-hostable |
|---|---|---|---|---|
| GLM 5.3 | 75.5 | $0.18/M (Z.ai) | Yes (MIT) | Yes (FP8 or NVFP4) |
| Claude Opus 5 | 76.0 | $15/M | No | No |
| GPT-5.5 | 76.8 | $12/M | No | No |
| Gemini 3.0 Pro | 76.4 | $10/M | No | No |
Suitable use cases
- Code agents in production. GLM 5.3 outpaces DeepSeek V4 Flash and Qwen 3.8 Max on SWE-Bench Verified (74.1%). On DGX Station GB300, the environment sustains 30 concurrent users in IDE mode.
- Long-context retrieval-augmented generation (RAG). 200K tokens cover a full 500-800 page A4 dossier. With a Qdrant index and a 128K practical window, it serves 18 queries per second at average load.
- Tool-augmented generation. Terminal Bench 2.1 at 74% indicates solid shell mastery, Navigation API, structured JSON. Compatible with OpenCode, Hermes Agent, internal harnesses.
- Long-form writing in French. French quality is slightly behind English but the GLM-2 tokenizer extends multilingual coverage. Reserve for content over 4,000 words.
When not to pick GLM 5.3?
- If the primary task is pure mathematical reasoning, Claude Opus 5 and GPT-5.5 still lead (AIME 80+).
- If the organization is strictly limited to a single DGX Spark (128 GB), Qwen 3.8 27B dense in FP8 fits on a single node without offload.
- If latency dominates over quality, a compact model (Qwen 3.8 7B, MiniMax M3 15B) delivers twice the speed on the same hardware.
- If input context regularly exceeds 200K, DeepSeek V4 Flash 0731 remains the only open-weight option.
Production deployment: checklist
- Pull BF16 weights from Hugging Face, ModelScope mirror, or the NVFP4 version from NVIDIA.
- Pick the runtime: vLLM 0.7+ for production, llama.cpp for debugging, SGLang for high-traffic agentic workloads.
- Compute the KV cache based on the target window. GLM 5.3 uses 64 KiB per token in FP8, against 256 KiB in BF16.
- Size hardware via the QDNA sheet of the chosen GPU (DGX Spark, DGX Station, H200, B200, B300, GB300 NVL72).
- Serve via OpenAI-compatible endpoint, hook into LiteLLM, add the semantic router if multiple models coexist.
- Measure throughput in single-user then load mode with Locust or k6. The p95 < 2s target is achievable on DGX Station and above.
- Connect monitoring: inter-token latency, NCCL RTT (if clustered), VRAM saturation, KV truncation rate.
Want to size GLM 5.3 on your hardware?
A half-day scoping workshop run on your own volume and your own use cases.
Book a callFrequently asked questions
Is GLM 5.3 open weight?
Yes. Weights are published by Zhipu AI (THUDM) on Hugging Face and ModelScope under the MIT license. The NVFP4 version is distributed by NVIDIA in partnership with Zhipu.
What hardware can run GLM 5.3 locally?
In FP8 on a DGX Spark x2 cluster (256 GB unified, 128K context, ~28 tok/s), on a DGX Station GB300 (288 GB, 200K context, ~45 tok/s), or on an H200 SXM node (141 GB, 200K context, ~95 tok/s). In NVFP4: GB300 NVL72 multi-node or B300 SXM x8 (200K context, 200+ tok/s).
What is the difference between GLM 5.3 and GLM 5.2?
GLM 5.3 pushes reasoning and agentic use: +2.3 points on Intelligence Index, +4 points on SWE-Bench Verified, and a better score on Terminal Bench 2.1. Total weight remains ~744B with ~20B active per token. Context window grows from 128K to 200K.
Is GLM 5.3 better than Claude Opus 5?
GLM 5.3 beats Claude Opus 5 on SWE-Bench Verified (74.1 vs 72) and Terminal Bench 2.1 (74.0 vs 70.2), trails it on MMLU-Pro (88.6 vs 89.4) and GPQA Diamond (76.2 vs 81.0). For open-weight use without third-party API, GLM 5.3 is today the strongest frontal option for self-hosting.