[audio.cpp] What Does the Fox Say: 4 ASR models (Nemotron 3.5 ASR, Higgs Audio STT, VibeVoice ASR, and Hviske ASR) in native C++/GGML, init streaming support, and 327s of audio transcribed in 2.17s.

I just pushed a new audio.cpp update with streaming support and 4 ASR models: Nemotron 3.5 ASR, Higgs Audio STT, VibeVoice ASR, and Hviske ASR (da only). Overall 1.07x to 2.41x faster than Python.

I decided to drop Parakeet-TDT since good implementations already exist, and I find the Nemotron model more interesting.

Streaming support is also starting to land in audio.cpp.

Right now, I added initial streaming support for two models that naturally fit streaming usage: Nemotron ASR and VoxCPM2. Nemotron ASR can stream recognition results through SSE, and VoxCPM2 can receive text incrementally and return generated audio chunks. I also experimented with streaming support for Higgs Audio STT, though I still treat that path as more experimental because the model/reference behavior is less straightforward.

There is still a lot to improve. The current work is only the first step toward proper model-level and server-level streaming support. Contributions are very welcome, especially around better streaming APIs, chunk scheduling, lower TTFT, server behavior, and adding streaming paths for more models.

The headline result: Nemotron ASR transcribed a 327.6s audio file in 2.17s offline on RTX5090 with 3.18% WER, and the streaming SSE path produced the same WER with 307ms TTFT and much lower peak VRAM. VibeVoice ASR gave the lowest WER on this test, but it is much heavier. Nemotron is the most interesting result to me because the speed/VRAM tradeoff looks very strong, especially for local ASR service usage.

ASR models (No Quant):

Streaming TTS:

Repo: https://github.com/0xShug0/audio.cpp

audio.cpp is still pretty new, but the goal is becoming clearer: a ggml-based local audio framework that can handle TTS, ASR, voice cloning, long-form generation, and server-like usage without every model needing its own Python environment and custom runtime.

As always, backend/OS compatibility cannot be fully tested by one setup. If you try the Windows, Linux, CUDA, Vulkan, Metal, or CPU paths and find issues, detailed reports are very welcome.

Huge thanks to our community members for their contributions!