ByteDance's Seed team has released SeedRealtime, a native audio-video full-duplex model that watches, listens, and speaks at the same time. Announced on August 5, 2026 and rolling out first inside Doubao, ByteDance's flagship assistant, rather than as a public API, it pushes real-time AI past the audio-only voice assistants creators already use and toward interfaces that react to a live camera feed the way a person in a conversation does.
Background
Most voice assistants today are half-duplex. You speak, the model waits for you to stop, then it transcribes, thinks, and speaks back. Full-duplex means the model listens and talks on overlapping channels, so it can react while you are still mid-sentence, pause when you interrupt, and pick the thread back up. Over the past year that pattern reached creators through the OpenAI Realtime API and through xAI's Grok Voice, but those stacks are built around audio.
SeedRealtime, described in the Seed team announcement, adds the visual channel as a first-class input. It processes a continuous stream of audio, video, and text at once and generates speech in the same loop, so the assistant can respond to what its camera sees and what it hears without waiting for a turn to end. ByteDance is shipping it inside Doubao first, a consumer app with a large installed base, rather than exposing weights or an endpoint.
Deep Analysis
Native Fusion Instead of a Stitched Pipeline
The usual way to build a live assistant is a cascade: a speech-recognition model turns audio into text, a language model reasons over the text, and a text-to-speech model reads the answer back. Each stage adds latency, and the hand-offs are where the awkward pauses and the talking-over-you failures come from. ByteDance says SeedRealtime instead uses a unified end-to-end architecture that fuses audio, video, and text natively, so there is no seam between hearing, seeing, and answering. The practical payoff is a model that can start forming a response from partial input and revise it as the input keeps arriving, which is what makes an interruption feel handled rather than ignored.
The Watch-and-Listen Leap
The more important shift is the camera. An audio-only assistant can hold a conversation, but it cannot see that you are holding the wrong cable, that your pan is smoking, or that your form is off. A native audio-video model can. ByteDance reports a 12 percent gain in conversational fluency over its previous models, crediting better listening and interference suppression so the model stops tripping over itself mid-sentence. Google's Gemini Live already accepts a live camera or screen feed, so SeedRealtime is not the first assistant that can see. What ByteDance is claiming is different: a single model trained to fuse the visual and audio streams rather than a voice stack with vision bolted on, which is the harder engineering problem and the one that decides whether watch-and-listen feels native or laggy.
How SeedRealtime Compares to the Real-Time Field
Real-time AI is quietly splitting into two questions: how many modalities the model handles at once, and how much of it a builder can actually touch today. SeedRealtime leads on the first and trails on the second.
| Model | Live modalities | Full-duplex | Access today | Openness |
|---|---|---|---|---|
| ByteDance SeedRealtime | Audio, video, text | Yes, native fusion | Doubao app only | Closed, no weights |
| OpenAI Realtime API / GPT-Live | Audio, text | Yes | Public API | Closed |
| Google Gemini Live | Audio, video input, text | Yes | Live API | Closed |
| xAI Grok Voice | Audio, text | Near real-time | Grok app and voice builder | Closed |
| NVIDIA VoiceChat-11B | Audio, text | Yes | Open weights | Open |
The table shows why SeedRealtime matters as a signal more than as a tool right now. It is the most complete on capability, fusing audio and video in one model, but it is the least accessible, locked inside a consumer app. The stacks a builder can prototype on today, OpenAI's and Google's, are the ones exposing an endpoint.
What a Builder Can Access Today
Because SeedRealtime has no public API or open weights, the move is not to wait for it but to build the interaction pattern now on the endpoints that exist. The OpenAI Realtime API gives you full-duplex audio with tool calls, and the Gemini Live API adds a live camera or screen feed so you can prototype the watch-and-listen loop, feeding frames alongside the audio. Design the product around continuous input and interruptible output, and you can swap in a native audio-video model like SeedRealtime the moment access widens, without rebuilding the interface.
Impact on Creators
Watch-and-listen changes what a live AI experience can be. A cooking walkthrough can correct your technique as it happens instead of narrating a fixed script. A hardware repair guide can watch your hands and tell you which screw is next. A language partner can read your expression and slow down when you look lost. On-camera tutoring, live product demos, and interactive streams all get an assistant that reacts to the scene rather than to a transcript. For anyone building these, the design lesson from SeedRealtime is to stop thinking in turns. Assume the model is always listening and always looking, and build the interface, the interruption handling, and the prompts around a conversation that never stops to wait.
Key Takeaways
- SeedRealtime is a native audio-video full-duplex model that watches, listens, and speaks in one continuous loop, announced August 5, 2026.
- Its edge is fusion: audio, video, and text handled by a single end-to-end model rather than a stitched speech-to-text-to-speech pipeline, with a reported 12 percent gain in conversational fluency.
- It is the most capable of the current real-time models on modality breadth but the least accessible, shipping only inside ByteDance's Doubao app with no API or open weights.
- Builders can prototype the same watch-and-listen pattern today on the OpenAI Realtime API and the Gemini Live API, then move to a native model as access widens.
What to Watch
The open question is distribution. If ByteDance ships a SeedRealtime API or open weights, it hands builders the first natively-fused audio-video assistant and forces OpenAI and Google to answer with more than a camera bolted onto a voice model. If it stays locked inside Doubao, it becomes a preview of where omni-modal interaction is going rather than a tool anyone outside ByteDance can build on. Either way, the direction is set: the next race in real-time AI is not who can talk fastest but who can watch, listen, and respond in a single unbroken loop, and the assistants creators reach for a year from now will be judged on whether they can see the scene, not just hear it.
