Introducing NVIDIA Nemotron 3 Super 🎉

Nemotron 3 Super: A Technical Deep Dive

Executive summary

• NVIDIA Nemotron 3 Super is a 120B-parameter (12B active) open hybrid Mamba-Transformer Mixture-of-Experts model with native 1M-token context, purpose-built for compute-efficient, high-accuracy multi-agent and agentic reasoning workloads.
• The architecture combines Mamba state-space layers for efficient long-sequence modeling with Transformer attention and a sparse MoE routing mechanism, delivering up to 5× higher throughput than dense Transformer baselines on agentic tasks.
• Fully open weights, training datasets, and reproducible recipes are released, enabling customization for domains such as software development, cybersecurity triage, and long-document analysis.
• The model follows the earlier Nemotron 3 Nano release and forms the flagship of the new Nemotron 3 family of open models, data, and libraries targeting transparent agentic AI development.

Technical architecture

Nemotron 3 Super employs a hybrid architecture that integrates three core components: Mamba-style state-space model (SSM) layers, traditional Transformer attention blocks, and a sparse Mixture-of-Experts (MoE) feed-forward network.

The total parameter count is 120 billion, but only 12 billion parameters are active per token due to the MoE design. This yields a roughly 10× reduction in active compute compared with a dense 120B model while preserving (and in many cases exceeding) the representational capacity of much larger dense models on agentic benchmarks.

The Mamba component is used strategically in the early and late layers of the network to handle very long contexts efficiently. Mamba’s selective state-space mechanism provides linear scaling in sequence length for recurrent-like memory, making native 1-million-token context feasible without the quadratic cost of full attention. Transformer attention layers are retained in the middle of the stack where global token-to-token dependencies are most critical for complex reasoning and tool-calling behavior.

Routing in the MoE layers is performed by a learned top-k gating network that selects a small subset of experts (typically 2–4 out of a larger expert pool) per token. NVIDIA has not disclosed the exact number of experts in the public announcement, but the 12B active parameter figure implies a total expert pool whose summed parameters reach the full 120B count.

The model is trained with a mixture of next-token prediction, agentic-specific objectives (including tool-use simulation and multi-turn dialogue), and synthetic data generated by earlier Nemotron models. NVIDIA is releasing not only the final weights but also the complete training dataset and the “recipes” (training code, hyperparameters, and data-mixing schedules) that allow the community to reproduce or further fine-tune the model.

Native support for 1M-token context is achieved through a combination of Mamba’s efficient recurrence, RoPE (Rotary Position Embeddings) extended to 1M, and careful attention-window management in the Transformer blocks. This makes Nemotron 3 Super particularly well-suited for long-document analysis, repository-scale code understanding, and multi-agent orchestration where agents must maintain extensive shared context across many turns.

Performance analysis

NVIDIA claims that Nemotron 3 Super delivers up to 5× higher throughput than comparable dense Transformer models of similar quality on agentic workloads. This throughput advantage stems directly from the 10× reduction in active parameters per token and the linear scaling properties of the Mamba layers.

While full independent benchmark numbers have not yet been released by third parties, NVIDIA’s internal evaluations highlight strong results on:

• Multi-agent collaboration and tool-calling tasks
• Long-context retrieval and reasoning (up to 1M tokens)
• Software engineering benchmarks (repository-level code understanding and generation)
• Cybersecurity triage and threat analysis scenarios

The model is positioned between the earlier Nemotron 3 Nano (a smaller, faster model released in December) and much larger dense models such as Llama 3.1 405B or Mixtral 8x22B in terms of capability, while offering significantly better inference economics than any of them.

Because the model uses only 12B active parameters, it can be served on fewer GPUs than a dense 70B or 120B model while still achieving higher effective quality on agentic tasks. This makes it attractive for both on-premise deployment and high-volume cloud agentic services.

Technical implications

The release of Nemotron 3 Super has several important implications for the AI ecosystem:

1. Agentic AI at scale — The combination of long context, MoE efficiency, and hybrid Mamba-Transformer design directly addresses the two biggest bottlenecks in multi-agent systems: context fragmentation and inference cost. Organizations building collaborative agent teams for software development, security operations, or enterprise automation now have an open model explicitly optimized for their workload pattern.

2. Openness and reproducibility — By releasing weights, full datasets, and training recipes, NVIDIA is lowering the barrier for researchers and enterprises to specialize the model for vertical domains. This mirrors the successful strategy used with earlier Nemotron models but at a significantly larger scale.

3. Hybrid architecture validation — The strong performance of a Mamba-Transformer hybrid at 120B scale provides empirical evidence that pure Transformer scaling is not the only viable path. It validates the growing industry interest in state-space and recurrent architectures for long-context and efficient inference.

4. Inference infrastructure impact — The model’s 12B active parameter footprint means it can run efficiently on current-generation NVIDIA GPUs (H100, H200, and Blackwell) with standard vLLM or TensorRT-LLM serving stacks, while the native 1M context will push developers to adopt new context-management and retrieval techniques.

Limitations and trade-offs

Despite its strengths, Nemotron 3 Super carries several limitations:

• MoE routing overhead — While active parameters are low, the total parameter footprint still requires substantial GPU memory for loading all experts. Loading the full 120B on a single node remains memory-intensive even if only 12B are used per forward pass.
• Maturity of hybrid stacks — Production inference frameworks have historically been optimized for pure Transformer architectures. While TensorRT-LLM and vLLM are adding Mamba and hybrid support, the ecosystem is still catching up, potentially leading to initial integration friction.
• Lack of independent benchmarks — As of the announcement date, only NVIDIA-provided numbers are available. The community will need to run comprehensive evaluations on AgentBench, LongBench, SWE-Bench, and other agentic suites to validate the 5× throughput claim across diverse workloads.
• Training data transparency — Although the dataset is being released, the exact mixture ratios and synthetic data generation pipelines used for agentic fine-tuning are complex; full reproduction may require significant compute resources.

Expert perspective

Nemotron 3 Super represents a pragmatic and technically sophisticated step in the evolution of open foundation models. Rather than chasing raw parameter count, NVIDIA has focused on architectural efficiency and workload-specific optimization for the emerging class of multi-agent and agentic systems. The hybrid Mamba-Transformer MoE design is one of the first large-scale public demonstrations that combining state-space efficiency with attention expressivity at MoE scale can deliver both performance and cost advantages.

The decision to open-source the complete training stack alongside the model is particularly significant. It transforms Nemotron 3 Super from a single artifact into a platform for experimentation, potentially accelerating progress in long-context modeling, efficient routing algorithms, and agent-specific pre-training objectives.

For ML engineers building production agentic systems, this model offers an immediately usable, highly efficient backbone that can be further customized without starting from closed-source APIs. The 1M-token native context removes a major pain point that has forced many teams to implement complex chunking and retrieval-augmented generation pipelines.

Technical FAQ

How does Nemotron 3 Super compare to Llama 3.1 405B or Mixtral 8x22B on agentic workloads?

NVIDIA claims comparable or better accuracy on multi-agent, tool-calling, and long-context tasks while using only 12B active parameters versus the 405B (dense) or ~39B active parameters of Mixtral 8x22B. The 5× throughput advantage is primarily versus dense models of similar quality. Independent verification on standardized agentic benchmarks is still pending.

Is the 1M-token context truly native or does it rely on extrapolation techniques?

The announcement states “native 1M-token context,” achieved through the Mamba layers’ linear scaling, extended RoPE, and selective use of attention. This differs from pure extrapolation methods used by some other models and should provide more stable performance at extreme lengths, though real-world testing at 1M tokens will be necessary.

What inference frameworks support the hybrid Mamba-Transformer architecture today?

TensorRT-LLM has added experimental Mamba support, and the vLLM team is actively integrating hybrid architectures. NVIDIA is expected to provide optimized inference containers and example serving code as part of the open release. Early adopters may need to use the provided recipes to compile custom kernels.

Can the model be further trained or distilled using the released datasets and recipes?

Yes. NVIDIA is releasing the full dataset and training recipes explicitly to enable continued pre-training, supervised fine-tuning, and distillation. The 120B total / 12B active design lends itself well to further MoE specialization or pruning experiments.

References

• NVIDIA Technical Blog: Introducing Nemotron 3 Super
• NVIDIA Blog: New NVIDIA Nemotron 3 Super Delivers 5x Higher Throughput for Agentic AI
• NVIDIA Newsroom: NVIDIA Debuts Nemotron 3 Family of Open Models

Sources

• NVIDIA Technical Blog — Introducing Nemotron 3 Super: An Open Hybrid Mamba-Transformer MoE for Agentic Reasoning
• NVIDIA Blog — New NVIDIA Nemotron 3 Super Delivers 5x Higher Throughput for Agentic AI
• NVIDIA Newsroom — NVIDIA Debuts Nemotron 3 Family of Open Models
• VideoCardz — NVIDIA releases Nemotron 3 Super, 120B open model with 1M-token context
• Original announcement on X by @NVIDIAAIDev