Interfaze, a project from JigsawStack, has released diffusion-gemma-asr-small, which it bills as the first open-source diffusion-based audio speech recognition model. Instead of transcribing audio one token at a time like Whisper and most modern systems, it starts with a canvas of random text tokens and refines them into a transcript through iterative denoising, the same diffusion idea that powers image generators, applied to speech. The full method is documented in Interfaze's launch write-up.
The headline accuracy numbers do not beat Whisper, and Interfaze is refreshingly honest about that. What makes this release matter is the architecture: it shows that a small trainable adapter can turn a frozen diffusion language model into a working transcriber, opening a research path that non-autoregressive ASR has been chasing for years.
What Diffusion ASR Actually Does
Autoregressive models like Whisper predict each word based on the words before it, left to right. A diffusion decoder works differently. It initializes the output as random vocabulary tokens, then runs a fixed number of denoising steps that progressively replace noise with the correct text, considering the whole sequence at once rather than strictly front to back. The appeal is speed and parallelism: you can trade denoising steps for latency, and the model does not have to generate sequentially.
diffusion-gemma-asr-small transcribes "through DiffusionGemma's own diffusion decoder," meaning it reuses Google's open diffusion language model as the text generator and teaches it to listen. This builds directly on the DiffusionGemma release we covered earlier.
How It Is Built
The architecture is deliberately lightweight. It combines a frozen Whisper-small encoder for acoustic features with a trainable adapter of roughly 42 million parameters, layered onto Google's frozen 26B DiffusionGemma backbone. Only the adapter is trained. The clever part is the training signal: the team used CTC loss to supervise the audio projector directly, bypassing attention early on so the model could establish a learning signal before the harder alignment work began.
Because only the adapter ships, the download is tiny. As Interfaze puts it, the release "ships only our ~42M-parameter adapter," while the frozen DiffusionGemma and Whisper backbones download from their own repositories under their own licenses. That makes the model cheap to distribute and easy to build on.
The Benchmarks, Honestly
| Model | Type | FLEURS English WER |
|---|---|---|
| Whisper-large-v3 | Autoregressive | ~4-5% |
| Whisper-small | Autoregressive | ~9-10% |
| diffusion-gemma-asr-small | Diffusion | 15.7% |
| Whisfusion | Diffusion (prior) | Higher WER, larger encoder |
On LibriSpeech test-clean the model records 6.6% word error rate, 15.7% on FLEURS English, and 18.5% on VoxPopuli English. Those trail Whisper. But the context matters: Interfaze reports the model was trained on roughly 219 hours of audio, versus about 680,000 hours for Whisper-small and around five million for Whisper-large-v3. Against the closest non-autoregressive comparison, it beats Whisfusion's 8.3% figure while using a smaller encoder, a result in line with recent research on diffusion-based non-autoregressive ASR. The takeaway is not "better than Whisper," it is "competitive per hour of training data, with room to scale."
Speed and Languages
The model runs at 3 to 5 times real-time speed with roughly 8 to 16 denoising steps, and step count is a tunable knob: fewer steps means faster transcription at some accuracy cost. It currently supports six languages, English, German, French, Spanish, Hindi, and Mandarin. That is narrower than the dozens Whisper covers, but consistent with a first release trained on limited data. For comparison on the production side, models like NVIDIA Nemotron ASR span 40 languages, which shows how much headroom remains.
How to Try It
- Visit the model card at interfaze-ai/diffusion-gemma-asr-small on Hugging Face.
- Clone the reference code from the JigsawStack GitHub repository.
- Let the setup pull the frozen DiffusionGemma and Whisper-small backbones, which download from their own repos under their own licenses.
- Point it at an audio file and set the denoising step count: start around 16 for accuracy, drop toward 8 for speed.
- Compare its output to Whisper on your own audio to judge whether the speed and architecture trade-offs fit your use case.
What This Enables
For most creators, Whisper and the current crop of production ASR models remain the right choice today for accuracy and language coverage. The reason to pay attention to diffusion-gemma-asr-small is what it signals. It is a working, open proof that diffusion language models can transcribe, that a 42M-parameter adapter is enough to bolt speech onto a frozen text diffusion model, and that the approach scales with data. If you build speech tooling, this is a reference implementation to study and fork now, before the accuracy gap closes. Diffusion ASR trades sequential decoding for parallel refinement, and that latency profile could matter for real-time transcription once the models mature.
Frequently Asked Questions
Is diffusion-gemma-asr-small more accurate than Whisper?
No. It posts a 15.7% word error rate on FLEURS English versus roughly 4-5% for Whisper-large-v3. It was trained on far less data, so the comparison is about efficiency and architecture, not raw accuracy.
What makes it different from normal ASR?
It uses diffusion decoding instead of autoregressive generation. Rather than predicting text one token at a time, it starts from random tokens and refines them into a transcript over several denoising steps, considering the full sequence in parallel.
How fast is it?
It runs at 3 to 5 times real-time with about 8 to 16 denoising steps. You can lower the step count for faster transcription at some cost to accuracy.
Which languages does it support?
Six: English, German, French, Spanish, Hindi, and Mandarin. That is narrower than Whisper, reflecting its limited training data.
How big is the download?
The release ships only a roughly 42 million parameter adapter. The frozen DiffusionGemma and Whisper-small backbones download separately from their own repositories under their own licenses.
Should I use it in production?
Not yet for accuracy-critical work. It is best treated as a research reference and a fork target for teams building diffusion-based speech tooling, rather than a drop-in Whisper replacement.
