Realtime-Venus posts state-of-the-art video and speech scores with full-duplex design

Realtime-Venus, a full-duplex multimodal agent described in a recent paper, has set new top scores across several video understanding and spoken dialogue benchmarks, according to a write-up published on dev.to. The system couples a 9-billion-parameter audio-visual model with a dedicated speech model, generates speech natively, and hands tool calls off to background processes so that computation does not stall the conversation.

How it differs from earlier agents

The dev.to post situates the work against systems such as Gemini 3.1 Live and GPT-4o, which it credits with strong performance in a single modality but criticises for not achieving smooth, simultaneous interaction across audio and visual streams. Earlier multimodal agents, the post notes, mostly ran half-duplex — listening or speaking, never both at once — or relied on post-hoc reasoning pipelines that froze the dialogue while processing in the background. Realtime-Venus instead perceives its inputs continuously while speaking.

Benchmark results

Two variants were evaluated. Realtime-Venus-Omni, the video-capable model, achieved the highest scores of the compared online models on six of eight video benchmarks, including 70.2% on StreamingBench, 64.7% on OVO-Bench and 81.3% on Daily-Omni, with consistent leads over prior online baselines across diverse streaming scenarios.

Realtime-Venus-Audio, the speech-focused variant, led the compared models on four tasks out of eight audio understanding and spoken question answering benchmarks: 78.0% on MMAU, 63.2% on MMAU-Pro, 83.8% on Llama Questions and 67.8% on Speech CMMLU. It also matched the best VoiceBench AlpacaEval score among the compared systems, at 4.81. Taken together, the post argues these numbers narrow the gap between models that only perceive and agents that must also hold up their end of a conversation.

Handling interruptions

On Full-Duplex-Bench v1.5, Realtime-Venus-Audio responded to 75% of user interruptions and kept its continuation rates at 97% under backchannels, 88% when other people in the scene were addressed, and 86% with background speech present — beating Gemini 3.1 Live and GPT-4o on all three continuation metrics, per the post. The upshot is that dialogue remains coherent even when a user talks over the system, which has historically been a weak point for voice agents built on turn-taking pipelines.

What remains untested

The evaluation covered only the 9B checkpoints in controlled benchmark environments. The post flags open questions around scaling to larger models, latency under real-world network variability, and robustness when confronted with visual scenes far from the training distribution. It also asks how the asynchronous tool delegation behaves when external APIs respond slowly or unpredictably, and whether the architecture's advantages would survive at a hundred-billion-parameter scale.

Why it matters

Most benchmark suites for multimodal agents still measure turn-based accuracy rather than the ability to be interrupted and carry on. If Realtime-Venus's numbers hold in production, evaluation practice will need to treat full-duplex interruption handling — how often a system reacts to a cut-in, and whether it can continue coherently afterwards — as a core criterion rather than an afterthought. Without it, progress in real-time voice and video agents risks being judged on proxies that miss conversational continuity, arguably the property users notice most.