On August 11, 2026, NVIDIA released Nemotron 3.5 Lightning, a 30-billion-parameter open-weights model built for always-on agents, together with NeMo Switchyard, an open-source router that decides which model should answer each step of an agent workflow. The pairing matters because it targets the two costs that keep agentic AI expensive: the price of every token an agent generates, and the price of sending every request to a frontier model when a smaller one would do. Lightning attacks the first with speed, Switchyard attacks the second with routing, and both ship with weights and source you can run yourself.
For builders, the headline is that you no longer have to choose between "fast and cheap" and "smart." You can run Lightning locally for the bulk of an agent's work and escalate only the hard steps to a frontier model, and NVIDIA is publishing the numbers to show what that saves.
What NVIDIA actually shipped
Two things dropped on the same day. The first is Nemotron 3.5 Lightning, a mixture-of-experts model with 30 billion total parameters and roughly 3 billion active per token. That active-parameter count is the whole point: the model reasons with the footprint of a 3B model while keeping the knowledge of a 30B one, which is how NVIDIA claims up to 4x the output speed of similar-sized models and 30 percent faster completion on long-running agentic tasks. It ships under the permissive OpenMDW-1.1 license, so commercial use is allowed.
The second is NeMo Switchyard, an open-source model-routing library. Instead of hard-coding one model into your agent, you register a pool of models and let Switchyard pick the cheapest one that can still hit your accuracy bar for each individual call. NVIDIA also released an agentic reinforcement-learning dataset, Nemotron-RL-Agentic-Terminal-Pivot, aimed at coding-agent behavior, so the training recipe is partly reproducible rather than a black box.
Nemotron 3.5 Lightning by the numbers
The model is distributed on Hugging Face as nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4, with NVFP4 and GGUF builds so it runs on both data-center and consumer stacks. Here is where it lands against the benchmarks NVIDIA published:
| Attribute | Nemotron 3.5 Lightning |
|---|---|
| Total parameters | 30B (mixture-of-experts) |
| Active parameters per token | ~3B |
| Output speed | Up to 4x similar-sized models |
| Agentic task completion | 30% faster than class peers |
| PinchBench accuracy | 86%, 10,000 tasks 30% faster than Qwen3.6 35B |
| License | OpenMDW-1.1 (permissive, commercial use allowed) |
| Formats | NVFP4, GGUF |
| Local runtimes | Ollama, llama.cpp, LM Studio, Unsloth, vLLM, SGLang, TensorRT-LLM |
The speed claim is the differentiator. Agents make many small model calls in sequence, so a 4x token-generation gain compounds across a task in a way a single benchmark score does not capture. That is why NVIDIA leads with "30 percent faster agentic task completion" rather than a raw accuracy figure: for an always-on agent, wall-clock time per task is the metric that shows up in your bill. The approach echoes what we saw with Meta Muse Glimmer, another open 30B model tuned for on-device agents, and it confirms 30B mixture-of-experts is becoming the default size for local agent work.
NeMo Switchyard: routing to cut agent cost
NeMo Switchyard is the more novel release. It sits in front of your model pool and evaluates each request at runtime against three signals: what each model is capable of, what each model costs, and how the underlying infrastructure is performing. It then routes the call to the cheapest model that still clears your policy, escalating to a frontier model only when the task demands it.
The published evaluations are specific. In a LangChain test pairing Nemotron 3.5 Lightning with Claude Opus 4.8 behind an escalation router, Switchyard cut cost 74 percent versus a frontier-only baseline while sending just 7 percent of calls to the frontier model, at a roughly 6-point accuracy tradeoff. Cognition's FrontierCode benchmark, routing between Opus 5 and Kimi K2.7, hit 50.6 percent accuracy at a $3.11 mean cost, within 2.8 points of Opus 5 alone at about 28 percent lower cost. Here is how those two routing setups compare:
| Routing setup | Frontier calls | Cost vs frontier-only | Accuracy tradeoff |
|---|---|---|---|
| Frontier-only baseline | 100% | baseline | baseline |
| Lightning + Opus 4.8 (LangChain escalation) | 7% | -74% | ~6 points |
| Opus 5 + Kimi K2.7 (Cognition FrontierCode) | varies | ~-28% mean cost | within 2.8 points |
Switchyard integrates with LangChain, LiteLLM, Kong, Cognition, Nous Research's Hermes Agent, Ramp, and Boomi's Agent Garden, so it slots into existing agent stacks rather than demanding a rewrite. That breadth is what makes routing practical: the router is only useful if it speaks the frameworks you already build on.
Run Nemotron 3.5 Lightning locally in four steps
Because Lightning ships in GGUF, you can have it answering prompts on a workstation or an RTX PC in minutes. Here is the fastest path:
- Pick a runtime. For a one-command start, use Ollama; for a graphical model browser with a built-in chat and local API server, use LM Studio; for maximum control over quantization, build from llama.cpp directly.
- Pull the weights. Grab the GGUF build from the Hugging Face repo. The NVFP4 build is the one to use on NVIDIA data-center hardware; the GGUF build is what consumer runtimes expect.
- Serve an OpenAI-compatible endpoint. Both Ollama and LM Studio expose a local HTTP server that speaks the OpenAI chat format, so any agent framework that talks to OpenAI can point at localhost with no code changes.
- Wire it into your agent. Swap your agent's base URL to the local endpoint and keep your existing tool definitions. If you already run coding agents against local models, the pattern is identical to running OpenCode against a local model in LM Studio.
What this enables for builders
The practical unlock is a two-tier agent that most teams could not economically run before. You point the agent at a local Lightning instance for the high-volume, repetitive steps, planning, tool selection, summarizing tool output, and let Switchyard escalate only the genuinely hard reasoning to a frontier model. On the published numbers, that keeps you within a few points of frontier accuracy while spending a fraction of frontier cost, and because Lightning is open-weights under OpenMDW-1.1, the local tier has no per-token fee at all.
It also changes what "always-on" is affordable. An agent that monitors a queue, watches a repo, or runs a nightly batch is prohibitively expensive on frontier tokens alone; a local Lightning tier makes continuous operation a fixed hardware cost rather than a metered one. NVIDIA's own framing, "always-on agents," is aimed squarely at that use case. This is the same open-weights, run-it-yourself direction as NVIDIA's Magpie TTS voice model, and it signals the company is serious about shipping usable weights, not just API access.
Frequently asked questions
Is Nemotron 3.5 Lightning really free to use commercially?
Yes. It ships under the OpenMDW-1.1 license, which NVIDIA describes as permissive and which allows commercial use. As with any model license, read the terms before shipping, but there is no API fee for running the weights yourself.
What hardware do I need to run it?
Because only about 3 billion parameters are active per token, Lightning is far lighter to run than its 30B total suggests. The GGUF build targets consumer runtimes like Ollama and LM Studio, so a modern RTX PC or a workstation with adequate VRAM can serve it. Exact requirements depend on the quantization you choose.
How is this different from just using a small model?
A small dense model is cheap but hits an accuracy ceiling. Lightning's mixture-of-experts design keeps 30B worth of knowledge available while only paying to run 3B per token, and Switchyard adds an escape hatch: when a step is too hard for the local model, it routes that one call to a frontier model instead of failing.
Do I need NeMo Switchyard to use Lightning?
No. Lightning is a standalone model you can run on its own. Switchyard is only worth adding when you want to mix a local model with a frontier model and let cost-versus-accuracy routing happen automatically. Many teams will start with Lightning alone and add routing later.
Which agent frameworks does Switchyard support?
At launch it integrates with LangChain, LiteLLM, Kong, Cognition, Nous Research's Hermes Agent, Ramp, and Boomi's Agent Garden. Because those cover the common agent and gateway layers, most existing stacks can adopt routing without a rewrite.
What does the "up to 4x faster" claim actually measure?
It refers to output token-generation speed against similar-sized models. NVIDIA pairs it with a 30 percent faster agentic task-completion figure, which is the more meaningful number for agents because they chain many calls together and wall-clock time per task drives cost.
What to do next
If you build agents, the fastest way to evaluate this is to pull the GGUF weights, serve them locally, and run one of your existing agent tasks against Lightning to see where accuracy holds and where it needs help. Then add Switchyard with a frontier model as the escalation target and measure your own cost curve rather than trusting the benchmark numbers. For a managed on-ramp without local setup, Lightning is also available on hosted inference platforms such as OpenRouter, which is a low-friction way to test the model before committing hardware to it.
