Nvidia releases Nemotron 3.5 Transcribe, an open 40-language ASR model with top leaderboard results

Nvidia has released Nemotron 3.5 Transcribe, an open-weight model that combines automatic speech recognition, speaker attribution, translation and audio question answering in a single checkpoint. According to Nvidia's blog post on Hugging Face, the model transcribes 40 languages, translates 31 of them to or from English, and attributes speakers in 13. It handles recordings up to an hour long, can label as many as eight speakers, and is available now on Hugging Face and ModelScope. The announcement surfaced on the front page of Hacker News.

The model is promptable: the same recording can yield a speaker-labelled transcript, an English translation, or an answer to a question such as what decisions were made in a meeting, depending on the natural-language instruction. Word-level timestamps are supported, which Nvidia demonstrates through a classroom demo from Agora that replays a recorded lesson alongside a synced transcript. Language coverage varies by task — audio question answering is trained in English only — and the full task-by-language matrix lives in the model card.

Leaderboard results

On the Hugging Face Open ASR Leaderboard, Transcribe ranks first among open-weight models for English with an average word error rate of 5.33%. Nvidia also reports 4.62% mean WER across seven languages on the multilingual track and 10.98% on long-form audio, the lowest figures among the open-weight models compared in the model card.

On Artificial Analysis's speech-to-text leaderboard, the model ties for first among open weights at 2.8% AA-WER while activating roughly 4 billion parameters, versus 24 billion for the model it ties with. Nvidia attributes the efficiency to pairing a speech encoder with a hybrid Mamba-Transformer mixture-of-experts language model, in which only about 4 billion of its 30 billion parameters are active per token. On AA-AgentTalk, a held-out voice-agent speech set, it scores 1.8% WER against 2.3% for the next-best open-weight model, and on the all-model chart it ranks ahead of several widely used proprietary speech APIs.

Speaker attribution is where Transcribe tops benchmarks outright. On David AI's DAI-ASR-I18N leaderboard it ranks first among open-weight models and second overall on the combined transcription-and-diarization track, and it posts the lowest diarization error rate of the 14 systems tested on the 21-language diarization track: 18.26%, narrowly ahead of Nvidia's own dedicated Nemotron 3 Diarization model at 18.38%. On VoiceArena's 14-system English benchmark it leads with 32.4% cpWER and drops far fewer words than its peers — deletions make up 8.32% of reference words against a median of about 31%.

Nvidia credits the diarization results to Sortformer speaker cues being fed into the language model together with the acoustic features, so speakers are tagged during generation rather than attached in a separate later pass.

Deployment and throughput

The released checkpoint can be served with vLLM today. Quantized versions are on the way, and Nvidia's benchmarking shows a forthcoming NVFP4 build processing 100 hours of audio in under three minutes on a single RTX PRO 6000 Blackwell Server Edition GPU, measured on 30-second clips at high concurrency. Local deployment through NeMo-Speech.cpp, targeting devices such as the DGX Spark and Jetson AGX Thor, is also planned. One limitation Nvidia flags: the model is built for recorded audio, not live streaming.

Granary v2 and fine-tuning

Alongside the model, Nvidia released Granary v2 — an open multilingual dataset and a reusable data-curation pipeline, also on Hugging Face and ModelScope — plus fine-tuning recipes built on the NeMo Speech tools. The idea is to extend prompt-level control into training, letting developers define the transcript conventions they need and adapt the model to their domain's vocabulary and accents. In a medical case study Nvidia cites, fine-tuning cut word error rate on held-out speakers by 45.8% relative to the best prompt-only result.

For unfamiliar terminology, the blog post suggests trying context before training: for example, listing candidate medical terms and instructing the model to use them only when the audio supports it. It also recommends spot-checking names, speaker changes and answers against the recording, as with any speech model.

Why it matters

Speech-to-text in production has leaned heavily on proprietary APIs, while open models have tended to trail on multilingual accuracy and speaker attribution — features that call analytics, meeting notes and captioning need as much as raw transcription. An open-weight model that leads the open leaderboards, sits on Artificial Analysis's accuracy-versus-active-parameters frontier, and ships with the same data pipeline used to build it gives developers a self-hostable alternative they can adapt to specialist vocabulary instead of working around a fixed API. The medical fine-tuning result suggests the gap between a general model and a domain-specific one can be closed with relatively little effort — assuming you have the GPU capacity to run and train it.