Nemotron 3 Ultra
NVIDIA
Nemotron 3 Ultra is NVIDIA’s open-weight mixture-of-experts model. It combines 550 billion parameters, of which 55 billion are active, in a Mamba 2 architecture paired with a transformer, in NVFP4 format, with a one-million-token context. It aligns with the NVIDIA AI Enterprise stack.
Key points: Open NVIDIA model · Mamba 2 plus transformer · NVFP4 · 1M context · NVIDIA AI Enterprise.
Architecture
Nemotron 3 Ultra combines 550 billion parameters, of which 55 billion are active (512 experts, 22 active per token), in a LatentMoE architecture that pairs Mamba 2, mixture of experts and attention, with multi-token prediction. The model card announces a context of up to one million tokens, while the config.json of the NVFP4 repository sets max_position_embeddings to 262,144. NVFP4 weights: 352 GB published; OpenMDW 1.1 licence (Hugging Face repository re-read on 2 September 2026).
Strengths
It stands as the reference open-weight model from NVIDIA and aligns with the NVIDIA AI Enterprise stack.
Use cases
Nemotron suits long-context workloads and deployments aligned with the NVIDIA ecosystem, with enterprise support.
Deployment
Served by vLLM or Triton in NVFP4 format on Blackwell GPUs, it takes advantage of native FP4 hardware acceleration.
All models on the platform are interchangeable through a single gateway: switching models is just a matter of changing one configuration line, with no code rewrite. Local execution remains the priority.
Official sources
- nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4, official model card on Hugging Face: OpenMDW 1.1 licence, parameters, context window and weight format
- Artificial Analysis, model comparison index
Which machine can run Nemotron 3 Ultra?
Nemotron 3 Ultra totals 550 billion parameters, about 316 GB in NVFP4. 7 platforms in the catalogue have enough memory. Active parameters govern speed, not footprint: every expert stays resident.
| Platform | Memory | Most precise format that fits | Sizing page |
|---|---|---|---|
| Mac Studio Ultra | 512 GB | NVFP4 | Nemotron 3 Ultra on Mac Studio Ultra |
| DGX Station | 748 GB | FP8 | Nemotron 3 Ultra on DGX Station |
| RTX PRO 6000 server | 768 GB | FP8 | Nemotron 3 Ultra on RTX PRO 6000 server |
| H200 SXM server | 1,128 GB | FP8 | Nemotron 3 Ultra on H200 SXM server |
| B200 SXM | 1,440 GB | FP16 | Nemotron 3 Ultra on B200 SXM |
| B300 SXM | 2,304 GB | FP16 | Nemotron 3 Ultra on B300 SXM |
| GB300 NVL72 | 20,700 GB | FP16 | Nemotron 3 Ultra on GB300 NVL72 |
See also Nemotron 3 Ultra in NVFP4 and the full sizing matrix.
Frequently asked questions
How many parameters does Nemotron 3 Ultra have?
550 billion in total, 55 billion of them active (512 experts, 22 active per token), in a LatentMoE architecture combining Mamba 2, mixture of experts and attention, with multi-token prediction.
What context length does Nemotron 3 Ultra accept?
The model card announces up to one million tokens; the config.json of the NVFP4 repository sets max_position_embeddings to 262,144, the value to use when sizing the KV cache.
Under which licence and at what size is Nemotron 3 Ultra published?
The NVFP4 weights are 352 GB, published under the OpenMDW 1.1 licence, Hugging Face repository reviewed on 2 September 2026; the model fits the NVIDIA AI Enterprise stack.