Nemotron 3 Ultra

NVIDIA

Open-weight model · runs on-premises · Published on 5 July 2026 · Updated on 2 September 2026

Nemotron 3 Ultra is NVIDIA’s open-weight mixture-of-experts model. It combines 550 billion parameters, of which 55 billion are active, in a Mamba 2 architecture paired with a transformer, in NVFP4 format, with a one-million-token context. It aligns with the NVIDIA AI Enterprise stack.

Key points: Open NVIDIA model · Mamba 2 plus transformer · NVFP4 · 1M context · NVIDIA AI Enterprise.

Architecture

Nemotron 3 Ultra combines 550 billion parameters, of which 55 billion are active (512 experts, 22 active per token), in a LatentMoE architecture that pairs Mamba 2, mixture of experts and attention, with multi-token prediction. The model card announces a context of up to one million tokens, while the config.json of the NVFP4 repository sets max_position_embeddings to 262,144. NVFP4 weights: 352 GB published; OpenMDW 1.1 licence (Hugging Face repository re-read on 2 September 2026).

Strengths

It stands as the reference open-weight model from NVIDIA and aligns with the NVIDIA AI Enterprise stack.

Use cases

Nemotron suits long-context workloads and deployments aligned with the NVIDIA ecosystem, with enterprise support.

Deployment

Served by vLLM or Triton in NVFP4 format on Blackwell GPUs, it takes advantage of native FP4 hardware acceleration.

All models on the platform are interchangeable through a single gateway: switching models is just a matter of changing one configuration line, with no code rewrite. Local execution remains the priority.

Official sources

• nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4, official model card on Hugging Face: OpenMDW 1.1 licence, parameters, context window and weight format
• Artificial Analysis, model comparison index