NVIDIA Launches Nemotron 3 Super: AI Powerhouse Revealed

NVIDIA Releases Nemotron 3 Super, 120B Open Hybrid Mamba-Transformer MoE Model

Key Facts

What: NVIDIA introduced Nemotron 3 Super, a 120B-parameter (12B active) open hybrid Mamba-Transformer Mixture-of-Experts model
Context Window: Native support for 1 million tokens
Design Focus: Built for compute-efficient, high-accuracy multi-agent and agentic AI applications
Availability: Fully open weights, datasets, and training recipes released for customization
Positioning: Follows the December release of Nemotron 3 Nano as part of the broader Nemotron 3 family

Lead

NVIDIA today released Nemotron 3 Super, a 120-billion-parameter open model with only 12 billion active parameters that combines Mamba and Transformer architectures in a Mixture-of-Experts design. The new model features native 1M-token context and is specifically engineered for complex multi-agent AI workloads such as software development agents and cybersecurity triaging, delivering maximum compute efficiency alongside high accuracy. As part of the Nemotron 3 family of open models, data, and libraries, the release provides developers with full access to weights, datasets, and training recipes to enable transparent and specialized agentic AI development.

Technical Architecture and Design

According to NVIDIA’s technical blog, Nemotron 3 Super is a hybrid Mamba-Transformer MoE model totaling 120 billion parameters while activating only 12 billion parameters per forward pass. This sparse activation approach dramatically improves inference efficiency compared to dense models of similar scale, addressing key limitations in running large-scale agentic systems that require both long context and sophisticated reasoning.

The model incorporates Mamba state-space architecture elements alongside traditional Transformer layers, creating a hybrid design optimized for the demands of agentic workflows. These workflows typically involve multiple specialized agents collaborating on complex tasks, often requiring repeated tool calling, long-document analysis, and multi-step reasoning.

NVIDIA positions the Super variant as the high-capability offering in the Nemotron 3 lineup, following the lighter Nemotron 3 Nano model released in December. The family is designed to power transparent, efficient, and specialized agentic AI development across industries.

Key Capabilities and Performance

The most striking technical specification is the model’s native 1 million token context window. This extended context capability makes Nemotron 3 Super particularly suitable for long-document analysis, codebases spanning hundreds of files, and complex multi-turn agent interactions that maintain extensive conversation or tool-use history.

NVIDIA claims the architecture delivers 5x higher throughput for agentic AI workloads compared to previous approaches, enabling more cost-effective deployment of sophisticated multi-agent systems. The model targets use cases including:

Software development agents
Cybersecurity triaging
Tool-calling applications
Long-document analysis
Collaborative multi-agent workflows

By maintaining only 12B active parameters despite the 120B total size, the model achieves significant compute savings during inference while preserving the representational power needed for high-accuracy reasoning tasks.

Open Source Commitment

In line with NVIDIA’s growing emphasis on open AI development, Nemotron 3 Super is being released with fully open weights, datasets, and training recipes. This comprehensive release allows developers and organizations to:

Fine-tune the model for domain-specific applications
Reproduce and understand the training process
Customize architectures for specialized agentic use cases
Build upon the model without proprietary restrictions

The release includes not only the model weights but also the datasets used in training and the detailed recipes for continued pre-training and post-training, enabling full transparency and customization.

Competitive Context and Industry Implications

NVIDIA’s release of Nemotron 3 Super arrives amid intense competition in the open foundation model space. The 120B-parameter scale with 12B active parameters positions it as a highly efficient alternative to dense models in the 70B-405B range that have dominated recent open model releases.

The hybrid Mamba-Transformer architecture reflects a broader industry trend toward exploring alternatives to pure Transformer designs for better efficiency at scale, particularly for long-context and agentic applications. Mamba-based architectures have gained attention for their linear scaling with sequence length compared to the quadratic scaling of traditional attention mechanisms.

For developers building production agentic systems, the combination of large total parameter count, low active parameter count, and native 1M context represents a potentially significant advancement in the price-performance ratio for sophisticated AI agents.

Availability and Ecosystem Support

The model is immediately available for download and experimentation. Early reports indicate it is already being hosted on platforms such as Nebius AI Studio, expanding accessibility beyond direct NVIDIA infrastructure.

Developers can choose their preferred GPU type and deployment architecture, with NVIDIA providing guidance on production deployment considerations. The open nature of the release means organizations can run the model on their own infrastructure or through various cloud providers supporting large-scale GPU workloads.

Impact on Developers and Enterprise AI

For developers working on agentic AI, Nemotron 3 Super offers a powerful new foundation model specifically optimized for the unique demands of multi-agent systems. The 1M token context window is particularly valuable for applications that must process large code repositories, extensive documentation, or maintain long agent interaction histories.

Enterprise users in software development, cybersecurity, and other knowledge-intensive domains may find the model’s combination of scale, efficiency, and openness particularly compelling. The ability to fully customize the model using provided datasets and recipes allows organizations to create specialized versions without starting from scratch.

The release strengthens NVIDIA’s position in the AI software stack by providing high-quality open models that complement its hardware offerings. Organizations investing in NVIDIA GPU infrastructure can now access optimized open models designed to run efficiently on that hardware.

What’s Next

NVIDIA has indicated that Nemotron 3 Super is part of a broader family of models and tools designed to accelerate agentic AI development. The company is expected to continue expanding the Nemotron 3 ecosystem with additional model variants, improved training techniques, and specialized libraries for multi-agent orchestration.

Developers interested in the model should monitor NVIDIA’s developer resources for updated benchmarks, fine-tuning guides, and integration examples. The full technical documentation and training recipes will likely reveal additional insights as the community begins experimenting with the model.

The release of comprehensive datasets alongside the model weights opens possibilities for further research into efficient MoE training techniques and hybrid Mamba-Transformer architectures.

Industry Significance

NVIDIA’s decision to open-source both the model and its training data represents a significant contribution to the open AI community. By providing not just weights but complete recipes, NVIDIA is enabling deeper understanding and innovation in hybrid architectures and efficient large-scale models.

As agentic AI systems become increasingly central to enterprise applications, models like Nemotron 3 Super that are specifically designed for multi-agent collaboration and long-context reasoning may help accelerate adoption of these technologies.

The 5x throughput improvement claimed for agentic workloads could substantially reduce the infrastructure costs associated with deploying sophisticated AI agents, potentially democratizing access to advanced agentic capabilities.

Introducing NVIDIA Nemotron 3 Super 🎉

Sources

Original Source

Related Topics

Comments