NVIDIA Nemotron 3 Diarization, released on 23 September 2026, is a free, 100M-parameter open-weight model that works out "who spoke when" for up to eight speakers. In Argmax's OpenBench run, published the same day, it posted the lowest average diarization error of six systems: 19%, against 20% for the paid pyannoteAI precision-3 API. That headline is accurate. It is also an average of 11 columns, and the columns disagree.
Read dataset by dataset, the free model beats the paid API on 4 of the 11, and its average lead rests on two of those, both AMI meeting sets. On a third, VoxConverse, NVIDIA's own model card lists the test split in its training data. If you cut podcasts, interviews or talk-show video, that changes which column you should be reading. Below: the full table, the parts of it that match recorded creator audio, and a local workflow for a speaker-labelled transcript, including a packaging trap in NVIDIA's own CLI that changed at 07:00 UTC this morning.
What NVIDIA shipped on 23 September
Speaker diarization is the step that splits a recording into turns and tags each one with a speaker label, so a transcript reads "Speaker 1" and "Speaker 2" instead of one unbroken block. Nemotron 3 Diarization is the successor to Sortformer v2.1, and the headline change is the speaker ceiling: eight instead of four.
- One checkpoint, four latency presets. The same weights run with a 30.4 second input buffer for recordings, or 1.04, 0.64 and 0.32 seconds for live audio.
- Arrival-ordered labels. The first voice the model hears becomes speaker 1, the next new voice speaker 2, and so on. The labels are anonymous: the model never knows who anyone is.
- Licence. OpenMDW 1.1, and the card states the model "is ready for commercial or non-commercial use".
- Speed. On an NVIDIA RTX PRO 5000, the offline preset processes audio 1,340 times faster than real time (4,385 times with torch.compile), which is roughly 2.7 seconds, or 0.8 seconds, per hour of audio before model loading.
NVIDIA's launch post leads with a first place on Voice Arena's Diarization-Bench: 14.72% error against 19.3% for the next system, across 12 systems and 139 English conversations totalling about 22 hours. NVIDIA itself notes those are initial results that may change. Three partners shipped on day one. Baseten hosts it as batch, streaming and diarized-transcription presets, with one RTX PRO 6000 handling 200 six-minute files per minute. Argmax Pro SDK 3 runs it on Mac and iPhone, and DigitalOcean lists it in its model catalogue.
Nemotron 3 Diarization vs pyannote, Deepgram and AWS
OpenBench is Argmax's open benchmark harness, and its diarization table is the only public source this week that runs Nemotron 3 alongside commercial APIs on the same audio. Argmax is a launch partner, so read it as a partner benchmark, but the code and dataset configurations are public. Nemotron 3, Sortformer v2, Deepgram nova-3 and pyannoteAI precision-3 were run on 22 September 2026; pyannote community-1 on 29 September 2025; AWS Transcribe on 17 February 2025. The local models ran on an M2 Ultra Mac Studio. Figures are diarization error rate (DER), the share of speaking time given the wrong label or missed. Lower is better.
| Dataset | What the audio is | Nemotron 3 | pyannoteAI precision-3 | pyannote community-1 | Sortformer v2 | AWS Transcribe | Deepgram nova-3 |
|---|---|---|---|---|---|---|---|
| AISHELL-4 | Mandarin meetings | 11% | 10% | 12% | 32% | 22% | 96% |
| AMI-IHM | English meetings, headset mics | 9% | 30% | 36% | 18% | not run | 36% |
| AMI-SDM | Same meetings, one room mic | 11% | 32% | 38% | 23% | not run | 44% |
| AVA-AVD | YouTube, in the wild | 45% | 34% | 48% | 60% | 61% | 67% |
| AliMeeting | Mandarin meetings | 18% | 11% | 23% | 21% | 42% | 95% |
| CallHome | Phone calls | 15% | 15% | 30% | 19% | 37% | 66% |
| DIHARD-III | Mixed domains | 13% | 14% | 22% | 21% | 36% | 37% |
| EGO4D | First-person video | 41% | 37% | 48% | 50% | 61% | 70% |
| Earnings-21 | Earnings calls with Q&A | 20% | 10% | 10% | 45% | 18% | 20% |
| MSDWILD | YouTube, multi-speaker | 23% | 22% | 33% | 33% | 40% | 65% |
| VoxConverse | YouTube interviews, debates, talk shows | 8% | 9% | 11% | 22% | 13% | 17% |
| Average | 19% | 20% | 28% | 31% | 37% | 56% |
Two footnotes from Argmax matter. Deepgram ran with language autodetection switched on, which is why it misses nearly all the speech on the two Mandarin sets. And AWS was never re-run on the newer AMI labels, so its average covers nine datasets rather than 11. Against its own predecessor the gain is real either way: Argmax puts Sortformer v2.1's average error 60% above Nemotron's.
Where the average lead comes from
Line Nemotron 3 up against the paid pyannoteAI precision-3 column alone. The free model is lower on four datasets: AMI-IHM (9% against 30%), AMI-SDM (11% against 32%), DIHARD-III (13% against 14%) and VoxConverse (8% against 9%). They tie on CallHome at 15%. The paid API is lower on the other six, and the gaps are not small: 10% against 20% on Earnings-21, 34% against 45% on AVA-AVD, 11% against 18% on AliMeeting.
Take the two AMI columns out and the ranking flips. Across the remaining nine datasets pyannoteAI precision-3 averages 18% and Nemotron 3 averages 22%, by our arithmetic on Argmax's published figures. Two things about AMI are worth knowing:
- The labels. OpenBench scores AMI against tighter forced-alignment annotations rather than the original segments. NVIDIA trained on AMI's train and development meetings using those same forced-aligned labels. The test meetings are separate, so this is not leakage, but Nemotron learned exactly the boundary convention that column rewards, and the commercial APIs did not.
- VoxConverse is a different matter. The model card's training table lists "VoxConverse v0.3, Development and test". OpenBench's VoxConverse configuration scores the test split, 232 recordings. The 8% result was measured on audio the model was trained on. Treat that cell as unscored.
None of this makes Nemotron 3 a weak model. It is the best free option in the table by a wide margin, 9 points ahead of pyannote community-1 on average, and far ahead of AWS and Deepgram. What it does not show is a free model beating the best paid API across the board. That depends on what you record.
Which column is your audio?
The datasets map onto creator formats more neatly than benchmark names suggest. Match your show to the nearest column before you switch anything.
- Two-person interview or podcast. CallHome is the closest public proxy, and it is a tie at 15%. NVIDIA's own table adds a detail worth knowing: on the two-speaker subset of CALLHOME, Nemotron 3 scores 5.98% against 5.68% for the old Sortformer model. NVIDIA flags this itself. For a two-voice show, the new model is not a clear upgrade on that measure.
- Panels and roundtables with five or more voices. This is where the eight-speaker ceiling pays off. In NVIDIA's tables, DIHARD-III recordings with five to nine speakers drop from 40.21% error to 27.58%, and NOTSOFAR1 meetings with five to seven speakers from 29.38% to 7.86%. Past eight voices the model has no channel left, and NVIDIA warns that speech can then be missed or given to the wrong speaker.
- YouTube talk shows and street interviews. With VoxConverse set aside, MSDWILD is effectively a tie (23% against 22%) and AVA-AVD a clear loss (45% against 34%).
- Webinars and long Q&A sessions. Earnings-21 is the closest match and the worst result for Nemotron against the paid API: double the error, 20% against 10%. Even the free pyannote community-1 scores 10% there.
Streaming presets cost speaker counts first
The latency presets are not free, and the cost shows up in speaker counting before it shows up in the error rate. NVIDIA's card reports speaker counting accuracy, the share of recordings where the model finds the right number of people. On NOTSOFAR1's single far-field microphone set, 160 meetings, moving from the 30.4 second preset to 0.32 seconds raises the error rate by 32%, from 11.00% to 14.53%. Over the same move, counting accuracy falls from 78% to 55%.
For a recorded episode there is no reason to accept that: use the offline-style preset and let the model see more context. The counting column deserves a look even at full context. On AMI's 16 test meetings, Nemotron 3 counts correctly in 14 (87.50%) against 15 (93.75%) for the old four-speaker model, while cutting the error rate by 41% to 48%. Small sample, but it is the kind of miss that merges two guests into one label, so spot-check the speaker count on your own audio.
Workflow: a speaker-labelled transcript on your own machine
NVIDIA's local route is NeMo-Speech.cpp, an Apache-2.0 C++ runtime with a nemo-speech command-line tool for Linux, macOS and Windows. The steps below follow its CLI guide.
- Check which diarizer your build has. The installer prefers a verified native release, and the only tagged release, v0.1.0 from 19 August, indexes a single diarizer: the four-speaker Sortformer v2. Nemotron 3 became the default diarizer in the main branch at 07:00 UTC on 24 September, and no tagged release carries it yet. Run
nemo-speech model list. Ifnemotron-3-diarizationis not listed, your binary predates it, and--diarizewill quietly use the four-speaker model. Build from the main branch (the build guide covers CPU, Metal, CUDA and Vulkan) or use the Python route in step 6. - Get a WAV file. The CLI accepts mono or stereo PCM WAV from 8 to 96 kHz and resamples it to the 16 kHz mono the model expects. For an MP3 or M4A export, convert first:
ffmpeg -i episode.m4a episode.wav. - Diarize.
nemo-speech diarize episode.wav --preset v3-offline --format rttm --output episode.rttmwrites speaker turns in RTTM, the standard turn-list format most diarization tools read. Thev3-offlinepreset uses larger chunks and caches; the default streaming pass is the one the guide recommends for long recordings, and the full-attention--offlinemode is capped at about 6.6 minutes of audio. - Transcribe with speakers.
nemo-speech transcribe episode.wav --diarize --jsonturns on word timestamps and puts a 1-based speaker number on every word in the JSON output. - Name the speakers. Because labels follow arrival order, whoever speaks first is speaker 1. If the host always opens the show, speaker 1 is the host in every episode, and one find-and-replace names them across the whole transcript.
- Python route. The model is supported in Hugging Face Transformers from source (
pip install git+https://github.com/huggingface/transformers), loaded withAutoModelForAudioFrameClassification, and the card's offline example prints each turn as a speaker with start and end times. - Test before you switch. Run a 10-minute clip of your own show through both Nemotron 3 and pyannote community-1, then listen to the handovers. Interruptions and cross-talk are where the two will disagree.
If you would rather not compile anything, NVIDIA's Hugging Face Space runs the model in the browser, and Argmax's Playground app runs it on a Mac or iPhone.
What each option costs
Nemotron 3 weights are free under OpenMDW 1.1, which allows commercial use; our earlier piece on NVIDIA's OpenMDW licence covers what the terms permit. The strongest free alternative, pyannote community-1, is CC-BY-4.0, which means attribution. The paid precision-3 model sits behind pyannoteAI's plans: a Developer tier at €19 a month that includes €19 of usage credit, and a Starter tier at €99 a month with €99 of credit.
Diarization only answers who spoke when; the words still come from a speech recognition model. If you are choosing that half of the pipeline too, our speech-to-text comparison covers the options, and NeMo-Speech.cpp pairs the diarizer with NVIDIA's own Nemotron and Parakeet recognizers in one command.
What to do next
For a two-host show or a panel of up to eight people recorded to a WAV, Nemotron 3 on the offline preset is the free default worth testing this week. Check that your nemo-speech build actually lists it first. For webinars, earnings-style Q&A, film audio or anything with more than eight voices, keep pyannote, paid or community, until your own clips say otherwise. Whichever you pick, name speakers by arrival order and spot-check the speaker count on every episode.
Frequently asked questions
What is Nemotron 3 Diarization?
It is NVIDIA's open-weight speaker diarization model, released on 23 September 2026. It has about 100M parameters, labels up to eight speakers including overlapping speech, and runs from one checkpoint in offline or streaming mode with input buffers from 30.4 seconds down to 0.32 seconds.
Is Nemotron 3 Diarization free for commercial use?
Yes. It is released under the OpenMDW 1.1 licence, and NVIDIA's model card states it is ready for commercial or non-commercial use. The NeMo-Speech.cpp runtime is Apache-2.0.
Is Nemotron 3 Diarization better than pyannote?
On average in Argmax's OpenBench it scores 19% error against 20% for the paid pyannoteAI precision-3, and it beats the free pyannote community-1 (28%) comfortably. Without the two AMI meeting sets, precision-3 leads 18% to 22%, and it is clearly ahead on earnings calls and in-the-wild video.
How many speakers can Nemotron 3 Diarization handle?
Up to eight, double the four of Sortformer v2.1. Above eight, NVIDIA warns that speech can be missed or assigned to the wrong speaker.
Does Nemotron 3 Diarization tell me who each speaker is?
No. It outputs anonymous labels in the order voices first appear, so speaker 1 is whoever talks first. Mapping labels to names is up to you, from show notes or a known opening order.
Can I run Nemotron 3 Diarization on a Mac?
Yes. NeMo-Speech.cpp builds with a Metal backend, Argmax ran its OpenBench numbers on an M2 Ultra Mac Studio, and Argmax Pro SDK 3 supports the model on Mac and iPhone. Check that your nemo-speech build lists the model, since the only tagged release predates it.
