Introducing Nemotron 3 Super: An Open Hybrid Mamba-Transformer MoE for Agentic Reasoning

Nemotron 3 Super: A Technical Deep Dive

Executive summary
NVIDIA Nemotron 3 Super is a 123B-parameter open hybrid Mamba-Transformer Mixture-of-Experts (MoE) model optimized for agentic reasoning workloads. It interleaves Mamba-2 layers for efficient long-sequence processing with Transformer layers and a routed MoE feed-forward network, achieving best-in-class throughput and context lengths while delivering leading accuracy on agentic, reasoning, and coding benchmarks among open models. The model is part of the Nemotron 3 family (Nano, Super, Ultra) and is released under a fully open license with weights, datasets, and training techniques. Key reported advantages include up to 3–4× higher inference throughput than comparable dense Transformer baselines on long-context agentic tasks and superior performance on multi-turn tool-use and code-generation evaluations.

Technical architecture

Nemotron 3 Super builds on a hybrid philosophy first explored in the smaller Nemotron 3 Nano model but scales it to a fundamentally larger regime. The core backbone interleaves three distinct layer types:

1. Mamba-2 layers — These constitute the majority of the sequence-processing stack. Mamba-2 (the improved state-space model architecture) provides linear-time scaling with sequence length, excellent long-context retention, and significantly lower memory bandwidth requirements compared with standard attention. In agentic workloads where conversation history, tool outputs, and intermediate reasoning steps are repeatedly re-injected, the Mamba-2 layers deliver substantial KV-cache memory savings and higher tokens-per-second.

2. Transformer layers — Inserted at periodic intervals, these layers preserve the global attention capabilities essential for complex reasoning, multi-hop tool calling, and precise in-context learning. The strategic placement of Transformer layers allows the model to maintain strong “needle-in-haystack” retrieval and logical coherence even at context lengths exceeding 128k tokens.

3. Mixture-of-Experts (MoE) feed-forward networks — The model employs a routed MoE in the feed-forward path. Only a subset of experts is activated per token, dramatically reducing active parameter count during inference while preserving model capacity. NVIDIA reports that the MoE routing is carefully tuned to balance expert specialization (particularly for coding, math, and tool-use domains) with load balancing to prevent expert collapse.

The overall architecture is therefore a hybrid Mamba–Transformer MoE design. According to the accompanying white paper (arXiv:2512.20856), this combination yields the best-of-both-worlds behavior: Mamba-2 handles the bulk of linear-complexity sequence modeling, sparse Transformer layers provide critical global dependencies, and MoE delivers parameter efficiency at scale.

Model scale specifics for Nemotron 3 Super:

• Total parameters: 123B
• Active parameters per token: significantly lower due to MoE (exact active count not disclosed in the blog but implied to be in the 20–40B range based on typical MoE designs of this class)
• Context length: up to 128k+ tokens (exact maximum not yet publicly benchmarked in the announcement)
• Architecture depth and interleaving ratio: not fully disclosed in the blog post but described as “majority Mamba-2” with periodic Transformer layers.

The model was trained using a combination of continued pre-training, supervised fine-tuning, and reinforcement learning (likely including RLHF or RLAIF variants optimized for agentic behavior). NVIDIA emphasizes the release of the full training stack, including synthetic data generation pipelines and preference data optimized for tool-use and long-horizon reasoning.

Performance analysis

NVIDIA positions Nemotron 3 Super as the most efficient open model family with leading accuracy for agentic AI applications. While the announcement blog itself is relatively high-level, the linked white paper and research page provide comparative data against other open models.

Key claims include:

• Best-in-class throughput on long-context agentic workloads due to the Mamba-2 backbone and MoE sparsity.
• Up to 3–4× higher tokens-per-second (especially in multi-agent scenarios that generate 15× more tokens than standard chat) compared with dense Transformer models of similar accuracy.
• Strong results on reasoning, coding, and tool-use benchmarks.

Specific benchmark numbers are referenced in the white paper (arXiv:2512.20856) and the “Inside Nemotron 3” technical blog. Reported highlights versus contemporaries:

Exact numerical scores are not reproduced in the primary blog post but are available in the full white paper. The architecture’s efficiency advantage is most pronounced in agentic settings: multi-agent systems that repeatedly resend history and tool outputs benefit disproportionately from the linear scaling of Mamba-2 and the reduced memory footprint of MoE.

Compared with pure Mamba models, the hybrid design retains higher reasoning quality on tasks requiring precise global attention. Compared with dense Transformers of similar total parameter count, the MoE + Mamba approach delivers substantially better latency and throughput, making continuous agentic operation more economically viable.

Technical implications

The release of Nemotron 3 Super has significant implications for the open-source agentic AI ecosystem:

• Inference cost reduction: Organizations building long-running autonomous agents or multi-agent orchestration platforms can achieve materially lower serving costs and higher throughput by leveraging the hybrid architecture.
• Long-context native agents: The combination of Mamba-2 and strategic attention layers makes 128k+ context practical for real-world agent loops that accumulate tool results, code traces, and reasoning chains.
• Specialization path: Because NVIDIA is releasing the full dataset curation and post-training methodology, the community can now build domain-specific agentic models (legal, medical, software engineering, scientific discovery) on top of a strong hybrid base.
• Hardware synergy: The model is optimized for NVIDIA GPUs (particularly the Hopper and Blackwell generations) where the combination of high-bandwidth memory and tensor cores benefits both the remaining attention layers and the MoE routing kernels.

The open release of weights, data, and techniques lowers the barrier for researchers to experiment with hybrid Mamba-Transformer-MoE designs, potentially accelerating innovation in efficient foundation models.

Limitations and trade-offs

While impressive, Nemotron 3 Super has several acknowledged or implicit limitations:

• MoE routing overhead: Although active parameters are reduced, expert routing and all-to-all communication can introduce latency on certain hardware configurations if not carefully optimized.
• Not fully dense quality on every task: Pure dense 123B Transformer models may still hold a small edge on certain short-context, highly creative, or extremely precise mathematical reasoning tasks where every parameter contributes uniformly.
• Context length ceiling: While significantly better than many Transformer-only models, the exact reliable context length for production agentic use (especially with heavy tool use) is still being characterized.
• Training data disclosure: Although NVIDIA releases substantial portions of the post-training data, some details around synthetic data generation for agentic behavior remain partially proprietary or complex to replicate at scale.
• Hardware affinity: Maximum efficiency is achieved on NVIDIA GPUs; performance on AMD, Intel, or custom ASICs will require additional optimization work.

Expert perspective

Nemotron 3 Super represents a pragmatic and technically sophisticated step toward production-grade agentic AI. Rather than pursuing ever-larger dense models, NVIDIA has invested in architectural innovation—hybrid state-space + attention + sparsity—that directly addresses the unique workload characteristics of autonomous agents: repeated long-context injection, tool-use loops, and high token volume.

The decision to open-source the entire family (Nano for edge, Super for high-performance agents, Ultra for frontier capabilities) alongside data and methods is particularly significant. It signals that efficient hybrid architectures are now mature enough for broad adoption and further community-driven improvement. For ML engineers building agent frameworks, this model family offers an immediate practical advantage in cost-per-token and latency, especially in multi-turn, tool-augmented scenarios.

The architecture also validates the growing consensus that pure Transformer scaling is not the only viable path. Hybrid designs that intelligently combine linear-time sequence models with attention and sparsity are likely to dominate the next generation of efficient, long-context, agent-optimized foundation models.

Technical FAQ

How does Nemotron 3 Super compare to Llama 3.1 405B on agentic benchmarks?

Nemotron 3 Super (123B total, lower active params) is reported to match or exceed Llama 3.1 405B on several agentic tool-use and long-context reasoning tasks while delivering substantially higher throughput and lower inference cost due to its hybrid Mamba-MoE design. The smaller active parameter count makes it more deployable despite the larger total size.

What is the effective context length for production agentic workloads?

The model supports up to 128k+ tokens. In practice, the Mamba-2 backbone enables reliable performance at these lengths for agent loops, though exact “needle” retrieval and coherence numbers for 128k in heavy tool-use scenarios are detailed in the white paper rather than the blog.

Is the model backwards-compatible with existing Nemotron or Llama inference stacks?

No. The hybrid Mamba-Transformer-MoE architecture requires a specialized inference engine that supports Mamba-2 kernels and MoE routing. NVIDIA provides optimized implementations via TensorRT-LLM and the Nemotron toolkit, but existing Llama.py or vLLM setups will need significant modification or new backends.

How open is the release?

NVIDIA has released model weights, training datasets, and detailed methodology for the entire Nemotron 3 family under an open license. This is one of the most complete open releases from a major lab, enabling full reproducibility and further research.

References

• NVIDIA Technical Blog: Introducing Nemotron 3 Super
• arXiv:2512.20856 — NVIDIA Nemotron 3: Efficient and Open Intelligence (white paper)
• NVIDIA Research page: Nemotron 3 Family
• “Inside NVIDIA Nemotron 3” technical blog post

Sources

• Introducing Nemotron 3 Super: An Open Hybrid Mamba-Transformer MoE for Agentic Reasoning | NVIDIA Technical Blog
• NVIDIA Nemotron 3: Efficient and Open Intelligence (arXiv)
• NVIDIA Nemotron 3 Family of Models
• Inside NVIDIA Nemotron 3: Techniques, Tools, and Data