NVIDIA Nemotron-3-Ultra 550B: A Frontier LLM for Complex AI Workflows

Introducing NVIDIA Nemotron-3-Ultra 550B-A55B

Released June 4, 2026, the NVIDIA Nemotron-3-Ultra 550B-A55B-BF16 is a frontier-scale LLM engineered for the most demanding reasoning, agentic, and long-context workloads. It packs 550 billion total parameters, but only 55 billion are active thanks to its LatentMoE architecture.

The model supports up to 1 million tokens of context and works across English, French, Spanish, German, Italian, Japanese, Korean, Hindi, Portuguese, and Chinese. A built‑in thinking mode can be toggled via the chat template. Designed for AI agents, RAG, and high‑stakes analytical tasks, it is available under the OpenMDW 1.1 license for both commercial and research use.

Hybrid LatentMoE and Multi‑Token Prediction

The Ultra model combines Mamba‑2, mixture‑of‑experts, and selective attention layers into a latent MoE hybrid. Tokens are projected into a smaller latent space for expert routing, improving accuracy per byte.

Multi‑Token Prediction (MTP) layers share weights across prediction heads, which boosts training signal quality and enables faster inference through native speculative decoding. During pre‑training, NVIDIA used an NVFP4 recipe — most linear layers store weights, activations, and gradients in 4‑bit floating point, while stability‑critical projections (latent, MTP, attention, embeddings) remain in BF16 or MXFP8. This balanced design delivers frontier efficiency without sacrificing accuracy.

Four‑Stage Training Pipeline

Training unfolded in four steps:

1. Pre‑training on ~20T tokens of crawled and synthetic data with the NVFP4 recipe.
2. Supervised fine‑tuning on math, code, tool‑calling, and long‑range retrieval data.
3. Reinforcement learning using asynchronous GRPO across math, code, science, and multi‑turn tool use; MTP accelerated rollout generation.
4. Multi‑Domain On‑Policy Distillation (MOPD) — teacher models guide learning on the model’s own rollouts, aligning behavior with what it will actually produce at inference time.

Pre‑training data cutoff is September 2025; post‑training data is fresh through May 2026. All datasets and environment code (Megatron‑LM, NeMo RL, NeMo Gym, Data Designer) are open‑source.

Benchmark Highlights

Nemotron‑3‑Ultra competes at the top of the LLM leaderboard. It excels in agentic coding, high‑level math, and extreme long‑context retrieval.

Full results and evaluation harness details are available in the technical report.

Deployment Quick Start

The BF16 checkpoint is a large model. For single‑node inference, 8× B200 GPUs (≈1.5 TB HBM) are recommended. Multi‑node setups can use H100/H200/GB200/GB300 clusters orchestrated with Ray v2. All configurations enable chunked prefill and MTP‑based speculative decoding (5 draft tokens). Below are the basic launch scripts.

# Set the IP for the head node in RAY_HEAD_IP
export RAY_HEAD_IP=
export RAY_PORT=6379
export RAY_ADDRESS=${RAY_HEAD_IP}:${RAY_PORT}

# Start Ray head node (vLLM/SGLang will run on this node)
ray start --head --node-ip-address=${RAY_HEAD_IP} --port=${RAY_PORT}

# Start Ray worker node(s)
ray start --address=${RAY_HEAD_IP}:${RAY_PORT} --block

# Verify Ray cluster is ready
ray status --address=${RAY_HEAD_IP}:${RAY_PORT}

export MODEL_CKPT=PATH/TO/MODEL/CHECKPOINT

docker run -d --name nemotron-ultra-vllm \
--gpus all \
--ipc=host \
--network=host \
--shm-size=16g \
--ulimit memlock=-1 \
--ulimit stack=67108864 \
-v $MODEL_CKPT:/model:ro \
-e VLLM_WORKER_MULTIPROC_METHOD=spawn \
-e SAFETENSORS_FAST_GPU=1 \
-e NVIDIA_TF32_OVERRIDE=1 \
-e VLLM_LOGGING_LEVEL=INFO \
vllm/vllm-openai:v0.22.0 \
/model \
--host 0.0.0.0 \
--port 8000 \
--served-model-name