deniz.in

Markets

Weather

Loading weather

· via dev.to (home feed)

k2-fsa open-sources OmniVoice, a zero-shot TTS model covering 600+ languages with 3-second cloning

The k2-fsa team has released OmniVoice, an open-source zero-shot TTS model covering more than 600 languages, with voice cloning from a 3-15 second clip and support for CUDA, Apple Silicon and Intel Arc GPUs.

k2-fsa open-sources OmniVoice, a zero-shot TTS model covering 600+ languages with 3-second cloning

A 600-language open-source speech model

The k2-fsa team has released OmniVoice, an open-source text-to-speech model that supports more than 600 languages, according to a post on dev.to. The team behind it is best known for k2 and icefall, two widely used speech-recognition toolkits. The post claims this is the widest language coverage of any zero-shot TTS model currently available, spanning major world languages as well as a long tail of regional ones.

Zero-shot here means the model needs no extra training per language. Rather than building a dedicated voice for each language, it generalizes across its training data and generates speech directly from input text.

Cloning and instruction-based voice design

The post describes two ways to control the output voice. The first is cloning: OmniVoice can copy a speaker's voice from a reference clip of just 3 to 15 seconds, short enough to build a custom narration voice from a fragment of audio. The second is voice design, where the user describes the desired voice in plain language — the example given is "male, British accent" — and the model synthesizes a matching voice without any reference audio.

Under the hood, OmniVoice is built on a diffusion language model rather than the autoregressive designs common in current TTS systems. According to the post, this approach delivers high audio quality together with faster inference than the autoregressive alternative.

Built to run locally

OmniVoice ships with a Python API and command-line tools. The examples in the post load the checkpoint from Hugging Face under the k2-fsa/OmniVoice name, generate audio from a text string paired with a reference WAV and its transcript, and write output at 24 kHz. A separate CLI flag covers instruct-based generation when no reference clip is available.

Hardware support is unusually broad for an open-source release: the post lists CUDA GPUs, Apple Silicon via MPS, and Intel Arc GPUs via XPU. That puts the model within reach of consumer hardware rather than restricting it to cloud infrastructure.

Release and ecosystem

Code and weights are published on GitHub and Hugging Face, and the release is backed by a paper, "OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models" (Zhu et al., arXiv:2604.00688). Community integrations have already appeared, the post notes, including a ComfyUI extension called ComfyUI-OmniVoice-TTS and a Home Assistant integration for running local TTS in a smart-home setup.

Why it matters

Most high-quality multilingual TTS today sits behind commercial APIs. An open-source model that clones a voice from a 3-second clip, covers more than 600 languages, and runs on a laptop changes the economics for developers: no per-character API fees, full control over data, and offline operation.

The language list is arguably the bigger story. Commercial systems tend to focus on a few dozen high-demand languages, leaving smaller ones with weak or missing voices. The dev.to post singles out Thai as an example, arguing that high-quality local synthesis is now feasible for such languages without any cloud dependency.

The caveat is consent. Cloning a voice from seconds of audio is precisely the capability that enables impersonation and fraud, and the post itself warns users to verify they have the rights to a reference voice before putting it to work. As with other open voice-cloning releases, the gains for accessibility and localization arrive attached to a real abuse surface that anyone deploying the model will need to manage.

  • #text-to-speech
  • #open-source
  • #voice-cloning
  • #machine-learning
  • #local-ai

Related posts