Introducing NVIDIA Nemotron 3 Super 🎉

Building Multi-Agent Coding Agents with NVIDIA Nemotron 3 Super

Why this matters for builders
NVIDIA Nemotron 3 Super is an open 120B-parameter (12B active) hybrid Mamba-Transformer Mixture-of-Experts model with native 1M-token context that lets you run complex, compute-efficient multi-agent systems for software development, tool-calling, and long-context reasoning.

The release gives builders fully open weights, training datasets, and reproducible recipes — removing the last major barrier to production-grade agentic workflows that previously required closed models or massive GPU budgets. You can now fine-tune, distill, or deploy a model explicitly designed for collaborative agents without worrying about usage limits or opaque reasoning chains.

When to use it

• You are building autonomous coding agents that need to maintain project-scale context (entire codebases, long design docs, multiple ticket threads).
• You want high-accuracy tool calling and multi-step reasoning while keeping inference cost low (12B active parameters deliver 5× higher throughput than dense 120B models).
• Your application involves long-document analysis, cybersecurity triage, or software engineering agents that must collaborate without losing coherence.
• You need full control: you plan to customize the model with your own data or integrate it into private inference stacks.

The full process

1. Define the goal

Start by writing a one-page spec. Good scope for a first project:

Goal: Build a persistent “Senior Engineer” agent that can:

• Ingest an entire GitHub repository (or large monorepo slice)
• Accept natural-language tasks
• Break them down, write code, run tests, and iterate
• Maintain conversation + codebase context across 100k+ tokens

Success criteria:

• Can complete a medium-complexity feature (e.g. add authentication flow) with <4 correction loops
• Average latency per agent turn <8 seconds on a single H100
• Works with open-source tooling (LangGraph, LlamaIndex, or CrewAI)

2. Shape the spec & prompt strategy

Nemotron 3 Super excels at long-context multi-agent orchestration. Design your system prompt and agent roles carefully.

Starter System Prompt Template (copy-paste ready):

You are Nemotron-3-Super, a senior software engineer with 15 years experience. 
You have access to the full codebase in context. 
Current date: {{date}}

Rules:
- Think step-by-step but be concise in final output
- Always use available tools when needed
- If you need more information, ask for clarification before coding
- Prefer well-tested, production-grade patterns
- After writing code, always suggest the exact test command to run

Available tools: read_file, search_code, edit_file, run_test, git_commit, etc.

Create specialized agent personas:

• Architect Agent (high-level planning, 1M context advantage)
• Coder Agent (focused on file-level changes)
• Tester Agent (validates output)
• Reviewer Agent (catches style, security, and edge cases)

3. Scaffold the project

Use a clean structure:

nemotron-coding-agent/
├── agents/
│   ├── architect.py
│   ├── coder.py
│   ├── tester.py
│   └── reviewer.py
├── tools/
│   ├── codebase.py      # vector + BM25 hybrid search over repo
│   ├── executor.py      # safe sandbox execution
│   └── memory.py        # long-term memory store
├── graph.py             # LangGraph workflow
├── config.py            # model endpoint + parameters
├── main.py
└── requirements.txt

Recommended stack:

• Inference: vLLM or NVIDIA NIM (Nemotron 3 Super has official NIM support)
• Orchestration: LangGraph (stateful graphs work beautifully with 1M context)
• Embeddings: Use the same Nemotron family embedding model when available or Nomic-Embed
• Vector store: LanceDB or PGVector for hybrid search over code chunks

4. Implement carefully

Key implementation tips for Nemotron 3 Super:

Inference configuration (vLLM example)

from vllm import LLM, SamplingParams

llm = LLM(
    model="nvidia/Nemotron-3-Super-120B",  # check exact HF path in docs
    tensor_parallel_size=4,                # 4×H100 recommended
    max_model_len=1048576,                 # 1M context
    enforce_eager=False,
    gpu_memory_utilization=0.9,
)

sampling_params = SamplingParams(
    temperature=0.3,
    top_p=0.95,
    max_tokens=4096,
    stop=["<|end_of_turn|>"]
)

Long-context retrieval strategy Because you have 1M tokens, you can be more aggressive with context. Best practice:

1. Keep the last 4 conversation turns fully in context
2. Add the top-8 most relevant code files/chunks (hybrid BM25 + vector)
3. Include the full task ticket + acceptance criteria
4. Add recent git diff if iterating on the same feature

This still leaves ~700k tokens of headroom for the model’s own reasoning.

Multi-agent graph node example (LangGraph)

def coder_node(state):
    messages = state["messages"]
    codebase_context = retrieve_relevant_code(state["task"], k=6)
    
    prompt = f"""Current task: {state['task']}
    
Codebase context:
{codebase_context}

Previous conversation:
{messages[-4:]}

Respond with a plan then code changes."""
    
    response = llm.generate([prompt], sampling_params)
    # Parse response for tool calls or direct code edits
    return {"messages": messages + [response], "next": "tester"}

5. Validate

Run a structured evaluation loop:

• Unit tests: Create 10 synthetic tickets of increasing difficulty (simple bugfix → new microservice endpoint → refactor with breaking change)
• Metrics to track:
  • Task completion rate (human-verified)
  • Average number of correction cycles
  • Tokens used per task
  • Latency per agent step
  • Cost per task (track active parameters advantage)

Use the model’s own Reviewer agent to score outputs on a 1-10 scale for correctness, security, and style, then correlate with human judgment.

Pro tip: Run the same evaluation against Claude 3.5 Sonnet and GPT-4o as baselines. Nemotron 3 Super should be competitive on coding tasks while being dramatically cheaper at scale.

6. Ship it safely

Production checklist:

• Deploy behind NVIDIA NIM or self-hosted vLLM with proper rate limiting
• Implement strict output validation + sandbox execution (never run untrusted code in production context)
• Add human-in-the-loop gates for any PR that touches security or core business logic
• Monitor context usage — even with 1M tokens, you can still hit the ceiling on very large repos
• Version your agent prompts and graph state — agent behavior drifts
• Log every agent decision for auditing (especially important for cybersecurity or regulated use cases)

Start by shipping an internal tool that only senior engineers can invoke, gather feedback for 2 weeks, then expand access.

Pitfalls and guardrails

### What if the model hallucinates file paths or function signatures?
Nemotron 3 Super still hallucinates when context is noisy. Solution: always pair generation with a retrieval step that returns real file content. Never trust the model to “remember” exact signatures without re-fetching the file in the same turn.

### What if latency is too high for real-time use?
Use speculative decoding (supported in vLLM) and lower temperature. Route simple tasks to a smaller Nemotron 3 Nano or distilled version. Reserve the full 120B Super for complex planning and long-context synthesis steps.

### What if 1M context makes the agent too verbose?
Explicitly instruct the model in the system prompt: “Be concise. Only output code blocks when ready to edit. Summarize analysis in <3 sentences.” Use structured output (JSON mode) for planning stages.

### What if fine-tuning is necessary for my domain?
NVIDIA released the full dataset and recipes. Start with LoRA on the 12B active parameters using NVIDIA’s NeMo framework. Focus on tool-calling and code-editing formats. The MoE architecture makes full fine-tuning more feasible than dense models.

What to do next

1. Deploy the baseline multi-agent coding loop this week
2. Run the 10-ticket evaluation suite and record metrics
3. Fine-tune a domain-specific version on your internal codebase + ticket history (start small)
4. Add memory layer (vector + graph memory) so the agent remembers past architectural decisions
5. Explore turning the Reviewer agent into an automated code reviewer that runs on every PR

Nemotron 3 Super is one of the first truly open models built from the ground up for agentic workloads. The combination of 1M context, hybrid Mamba-Transformer efficiency, and full openness creates a new baseline for what autonomous coding systems can look like in 2025.

Sources

• Original announcement: https://x.com/NVIDIAAIDev/status/2031774913544016179
• NVIDIA Technical Blog: “Introducing Nemotron 3 Super: An Open Hybrid Mamba-Transformer MoE for Agentic Reasoning”
• NVIDIA Blog: “New NVIDIA Nemotron 3 Super Delivers 5x Higher Throughput for Agentic AI”
• NVIDIA Newsroom: “NVIDIA Debuts Nemotron 3 Family of Open Models”
• Additional coverage: VideoCardz, Nebius AI Studio hosting notes

(Word count: 1,248)