NVIDIA Guide: Install Git LFS Easily & Quickly

How to Get Started with NVIDIA Nemotron 3 Super for Multi-Agent Applications

TL;DR

Download the fully open 120B-parameter (12B active) hybrid Mamba-Transformer MoE model with native 1M-token context from Hugging Face and run it locally or in the cloud.
Use the provided datasets and training recipes to customize the model for your own agentic workflows such as software development agents or cybersecurity triage.
Deploy it today on supported inference platforms to achieve high throughput and compute efficiency for complex multi-agent systems.

Prerequisites

A machine or cloud instance with substantial GPU memory (minimum 8x H100 or equivalent recommended for full 120B inference; smaller setups possible with quantization).
Python 3.10+ environment with Hugging Face transformers and vllm or NVIDIA TensorRT-LLM installed.
Git and git-lfs for downloading large model files and datasets.
Basic familiarity with running large language models and building agent frameworks (LangChain, LlamaIndex, or custom multi-agent orchestration).
Hugging Face account with access to gated repositories (apply for access if required).

Step 1: Download the Model Weights, Datasets, and Recipes

NVIDIA has released Nemotron 3 Super with fully open weights, training datasets, and reproducible recipes.

Visit the official model card on Hugging Face (search for “Nemotron-3-Super-120B” or follow the link from the NVIDIA blog).
Accept the license and use the following commands:

git lfs install

# Clone the model repository (120B total parameters, 12B active)
git clone https://huggingface.co/nvidia/Nemotron-3-Super-120B

# Download the accompanying datasets and training recipes
git clone https://huggingface.co/datasets/nvidia/Nemotron-3-Super-Dataset
git clone https://huggingface.co/datasets/nvidia/Nemotron-3-Recipes

The model uses a hybrid Mamba-Transformer Mixture-of-Experts architecture. Only 12B parameters are active per forward pass, which delivers the 5× higher throughput mentioned in NVIDIA’s benchmarks compared to dense 70B-class models while maintaining or exceeding accuracy on agentic reasoning tasks.

Step 2: Set Up the Inference Environment

Choose your deployment path based on available hardware.

Option A: Local / On-Prem with vLLM (recommended for quick testing)

pip install vllm==0.6.3.post1

# Launch OpenAI-compatible server with 1M context support
python -m vllm.entrypoints.openai.api_server \
  --model ./Nemotron-3-Super-120B \
  --tensor-parallel-size 8 \
  --max-model-len 1048576 \
  --enable-mamba \
  --dtype bfloat16

Option B: NVIDIA TensorRT-LLM for maximum efficiency

Follow the official TensorRT-LLM examples in the recipe repository. The hybrid Mamba-Transformer design allows significant memory savings and faster decoding, critical for multi-agent systems that generate many tool calls and long reasoning traces.

Option C: Cloud platforms

Nebius AI Studio now hosts Nemotron 3 Super — sign up at nebius.com and select the model from their Token Factory.
Other supported platforms include major GPU cloud providers that offer H100/A100 clusters.

Step 3: Run Your First Inference with 1M-Token Context

Test the native 1-million-token context window immediately:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="sk-no-key-required"
)

response = client.chat.completions.create(
    model="Nemotron-3-Super-120B",
    messages=[
        {"role": "system", "content": "You are an expert software engineering agent."},
        {"role": "user", "content": "Here is a 900k-token codebase dump... Analyze the architecture and propose a refactoring plan."}
    ],
    max_tokens=4096,
    temperature=0.7
)

print(response.choices[0].message.content)

This demonstrates the model’s ability to process extremely long contexts — ideal for analyzing entire repositories, long security logs, or multi-document enterprise knowledge bases.

Step 4: Customize the Model Using Provided Recipes

The release includes full training recipes and datasets, making customization practical.

Navigate to the cloned Nemotron-3-Recipes folder.
Follow the supplied SFT (Supervised Fine-Tuning) and preference optimization scripts:

cd Nemotron-3-Recipes
pip install -r requirements.txt

# Example: Continue pre-training or SFT on your domain data
python train.py \
  --model_path ../Nemotron-3-Super-120B \
  --dataset_path ../Nemotron-3-Super-Dataset/custom_domain.jsonl \
  --output_dir ./nemotron-super-finetuned \
  --num_epochs 2 \
  --learning_rate 1e-5

Because only 12B parameters are active, fine-tuning is far more memory-efficient than a dense 120B model. NVIDIA reports strong results on software development and cybersecurity triaging tasks after domain-specific adaptation.

Step 5: Build Multi-Agent Applications

Nemotron 3 Super was purpose-built for compute-efficient, high-accuracy multi-agent applications. Here’s a minimal multi-agent setup:

from langchain_core.agents import AgentExecutor
from langchain_nvidia_ai_endpoints import ChatNVIDIA  # or your local endpoint

llm = ChatNVIDIA(model="Nemotron-3-Super-120B", temperature=0.3)

# Define specialized agents
planner = llm.bind(system="You are a task decomposition agent...")
coder = llm.bind(system="You are an expert Python engineer...")
reviewer = llm.bind(system="You are a senior security and code quality reviewer...")

# Simple multi-agent orchestration loop
task = "Build a secure REST API for processing sensitive health data"
plan = planner.invoke(task)
code = coder.invoke(plan)
final_review = reviewer.invoke(code)

The model’s hybrid architecture excels at tool-calling consistency and long-horizon reasoning required when multiple agents collaborate over extended sessions.

Tips and Best Practices

Start with 4-bit or 8-bit quantized versions (available or easily created with bitsandbytes or gptq) if you don’t have an 8×H100 cluster. The MoE design retains accuracy surprisingly well under quantization.
Leverage the 1M context for RAG over massive document collections — feed entire codebases or years of security logs in a single prompt.
Monitor active parameter usage; the 12B active design delivers the advertised 5× throughput improvement on agentic workloads.
Use the released datasets to create high-quality synthetic training data for your vertical.
Combine with NVIDIA’s NeMo Framework for enterprise-grade distributed training and serving if scaling beyond a single node.

Common Issues

Why am I getting "context length exceeded" errors?
Make sure you set --max-model-len 1048576 (or 1M) in vLLM/TensorRT-LLM. Many default configurations still use 128k or 32k.

Why is inference slower than expected?
Verify you enabled Mamba kernels and are using the correct tensor-parallel size for your GPU count. The hybrid architecture requires specific kernels for optimal Mamba-Transformer switching.

Model fails to download or says "gated repo"?
Create a Hugging Face account, accept the Nemotron license, and request access on the model card. Approval is usually granted quickly for open models.

Out-of-memory during fine-tuning?
Use the provided LoRA/DoRA recipes in the official repository. The 12B active parameter design makes parameter-efficient fine-tuning very practical.

Next Steps

After running your first multi-agent workflows, explore:

Full parameter-efficient fine-tuning on your proprietary data.
Integration with NVIDIA NIM (NVIDIA Inference Microservices) for production deployment.
Building agent teams specialized in software engineering, cybersecurity, or legal document analysis.
Benchmarking against other open models on long-context agentic tasks using the released evaluation recipes.

The combination of open weights, 1M context, and efficient MoE design makes Nemotron 3 Super one of the most immediately usable large models for production agentic AI today.

Install git-lfs if not already installed

Step 1: Download the Model Weights, Datasets, and Recipes

Step 2: Set Up the Inference Environment

Step 3: Run Your First Inference with 1M-Token Context

Step 4: Customize the Model Using Provided Recipes

Step 5: Build Multi-Agent Applications

Tips and Best Practices

Common Issues

Next Steps

Sources

Original Source

Related Topics

Comments