RealtimeVAD cuts end-of-speech latency by ~100 ms versus Silero VAD

A faster end-of-speech signal

A developer posting as KoljaB has released RealtimeVAD, an open-source Python library that tells a voice assistant its user has stopped talking a median 100 ms sooner than the commonly used Silero VAD. The project debuted on Hacker News as a Show HN post pointing to its GitHub repository, and it targets a bottleneck every realtime voice pipeline shares: recognition, the LLM call and audio output all wait on the single decision that the turn has ended, and a conventional detector can take a few hundred milliseconds to make it.

How it works

RealtimeVAD does not replace Silero; it keeps the model for deciding what counts as speech and layers a second, cheaper check on top. Every 10 ms it compares the incoming audio level against the room's noise and the speaker's own level, and emits a 'pause' event within about 10 ms of the audio going quiet — long before Silero's speech probability has fallen. A 'start' event fires after roughly 64 ms of confident speech so clicks do not trigger it, and 'end' closes the segment 0.24 s after speech-off unless speech resumes, never later than Silero alone would. Input is 16 kHz mono audio in chunks of any size (float32, int16 or int16 bytes). The library runs on one CPU thread faster than realtime, needs only numpy and onnxruntime, and bundles the Silero model so torch is not required.

The measured trade-off

According to the project's README, the author benchmarked 3,279 turns from a live voice assistant, computing Silero's decisions from the same frame probabilities so both detectors saw identical audio. Speech-off arrived a median 100 ms before Silero's probability fell below its default 0.5 threshold (earlier in 98% of turns), 110 ms before a stricter 0.35 threshold, and 206 ms before Silero's VADIterator reported the end. In the 2,323 turns where both closed the same segment, 'end' beat Silero's in 82% of cases by a median 70 ms and was never later.

The price is false alarms. 'pause' fires about ten times as often as Silero dipping below 0.5 — 1.5 events per second of speech versus 0.14 — because it triggers on every breath, stop consonant and short gap between words. That is deliberate: the README frames 'pause' as a speculative signal for work you can cancel, such as finalizing the transcript, calling the LLM or preparing voice output, aborting if speech continues. Pipelines that cannot cancel cheaply are told to ignore 'pause' and wait for 'end'.

Integration and caveats

For existing Silero users the library is positioned as a drop-in: swap the import, and the constructor and results keep the same shape, with 'end' simply arriving sooner. A LiveKit Agents plugin swaps silero.VAD.load() for realtimevad.VAD.load(); there, end of speech came about 380 ms sooner than the Silero plugin's defaults, though the author attributes most of that to a shorter silence setting (0.25 s versus 0.55 s) that Silero can be given too. At equal settings the end was 65–110 ms earlier and the turn committed about 70 ms sooner.

The project is at version 0.0.5 and labelled alpha, with an API that may still change. Quiet input is the documented weak spot: a quiet speaker's soft word endings are not silence, and on a noisy quiet line the lead disappears. Earlier versions used a fixed noise floor and cut quiet speakers off — at −52 dBFS, version 0.0.4 ended the segment while the user was still talking in 43% of turns — so 0.0.5 makes the floor track the speaker's own level, bringing that error rate down to near Silero's.

Why it matters

End-of-speech latency is one of the most user-visible costs in a voice assistant, and it sits in series with recognition, inference and synthesis, so every saved millisecond lands directly in perceived responsiveness. RealtimeVAD's contribution is not a better acoustic model but a sharper engineering trade: accept frequent, cheap 'maybe done' hints and let the pipeline speculate while speech may still resume. For builders of realtime voice agents, roughly 100 ms per turn is a real win, and the claim is unusually verifiable — both detectors run on the same frame probabilities, and the repository ships a benchmark script to reproduce the numbers on your own recordings.