Gradium open-sources Phonon on-device TTS engine; the hard part is real-time audio

Gradium has open-sourced Phonon, its on-device text-to-speech engine. Writing on dev.to, Dima Kruglov argues the release matters less for the underlying model than for what it signals: speech synthesis can now ship as a dependency rather than live as a lab demo, and the teams adopting it will find that audio, not the model, is where the difficulty sits.

Audio runs on a clock that does not wait

Web traffic tolerates slowness; audio playback does not. Once a stream starts, the digital-to-analogue converter consumes samples at a fixed rate, and the synthesis loop has to stay ahead. Fall far enough behind and the output buffer runs dry, producing clicks, pops or a robotic stutter that listeners read as a malfunction. The arithmetic, per Kruglov: at 24 kHz with a 20 ms mixer period, each callback must deliver 480 samples — continuously, even while the CPU throttles under heat and the OS runs a background sync.

That changes what gets optimized. Median latency is close to meaningless, the article argues. What counts is worst-case chunk time, because one slow chunk becomes an audible artifact; real-time factor sustained over minutes rather than seconds, because CPU governors ramp down; and slack, the gap between the worst chunk and the audio already pre-rolled into the buffer.

Real-time factor is an average, and averages hide failures

Real-time factor — synthesis time divided by audio duration — is the figure everyone quotes, but on its own it misleads. The author recounts an engine holding an RTF of 0.55 for a minute and drifting to 1.1 by minute four, after the SoC moved work from its big cores to its little cores and never migrated back. A five-second RTF measurement would have missed that entirely. The article ships a Python harness that runs ten-plus minutes on real hardware, reporting mean RTF alongside 99th-percentile and worst-case chunk times and whether buffer slack stays positive — the point being that a mean under 1.0 is necessary but not sufficient.

Prebuffering buys smoothness and costs responsiveness

Jitter is not fixed with a faster model alone; it is fixed with buffer. Pre-rolling 300–500 ms of audio before the output stream starts absorbs a late chunk without an underrun. The prebuffer is user-facing, though: in a voice assistant it is the pause between the user finishing a sentence and the assistant replying; in a screen reader it is lag on every cursor movement. Kruglov recommends tuning per surface: 150–250 ms for conversational products that accept occasional degradation, 400–800 ms for narration and media where nobody notices, and fully synthesizing short utterances before playback in accessibility tools, since a three-word phrase rendered in one pass costs less than one streamed in pieces.

Seams, quantization and platform-specific traps

Streaming TTS splits both text and audio, and both splits create seams. Sentence-level chunking shifts prosody across a join; frame-level vocoder chunking produces discontinuities unless chunks overlap and crossfade. Five to ten milliseconds of constant-power crossfade hides most of it — under 3 ms leaves audible ticks on plosives, over 20 ms smears consonants. Two failure modes stand out: text splitters cutting inside a token when a user types "e.g." or a German compound, and crossfades applied across natural pauses, which shrink silences and make the delivery feel hurried.

Quantization damages naturalness before intelligibility, the article continues: word error rate through an ASR round-trip can stay flat while the voice turns flat and slightly buzzy. Kruglov instead tracks F0 contour drift against a float32 reference over a fixed 50-sentence set, spectral distance on sustained vowels, and A/B listening on both phone speakers and earbuds. Threads trade off too: four threads finish chunks faster but draw more power and heat the SoC, feeding the throttling behind the RTF drift, so two threads plus a larger prebuffer often wins across a ten-minute session.

The platform notes are concrete. Android's low-latency AudioTrack path expects the device's native sample rate, so generating at 24 kHz and letting the framework resample to 48 kHz adds latency and an unasked-for filter. On iOS, low-latency routing depends on AVAudioSession category and mode and needs testing on hardware rather than the simulator. On embedded Linux, ALSA period sizes and start thresholds vary by board and are frequently wrong by default. Memory-mapping large model weights is the right call, but the first utterance after a cold start can fault in enough pages to miss deadlines unless the model is warmed at launch or the hot region is locked.

Why it matters

With Phonon in the open, on-device speech synthesis becomes something teams install rather than build. The bottleneck moves from whether the model can run on a phone to whether the output stays continuously supplied for ten minutes on a hot, throttled device. Engineers arriving from request-driven backend work, where slowness merely degrades an experience, will need new habits: long-session distributions instead of short averages, worst-case and slack metrics, and a degradation ladder chosen before launch rather than mid-incident. The argument applies to any edge audio engine, not only Phonon: the hard real-time deadline, not model quality, separates a demo from a shippable product.