Microsoft ships a streaming ASR model that already knows who is talking

What happened

Microsoft Research published VibeVoice-ASR-Streaming-7B to the Hugging Face hub on 2 September 2026. It is a streaming automatic-speech-recognition model that transcribes who said what in real time, covers ten languages, accepts user-supplied hotwords for domain-specific vocabulary, and is distributed under the MIT licence.

Context

Most open-weight ASR models transcribe; speaker diarisation is a separate model and a separate failure mode. VibeVoice-ASR-Streaming folds both into one pass: the model card describes it as a "unified streaming ASR model that transcribes who (speaker) said what (content)." It is a Microsoft Research project built on the transformers library with safetensors weights. The model card points to a technical report at arXiv:2609.02812 for the full architecture and evaluation. At collection time the model had 44 likes and 8 downloads, consistent with a fresh release.

How it works

The model processes audio in a streaming fashion: as frames arrive, it emits transcribed text tagged with a speaker label, without waiting for a full clip. It covers ten languages — Chinese, English, French, German, Italian, Japanese, Korean, Portuguese, Russian, Spanish — and accepts a user-supplied hotword list so that product names, character names, or domain jargon are recognised rather than guessed at. Integration is through the standard transformers pipeline; the endpoints_compatible hub tag suggests it can be served through an inference endpoint, though the model card does not name a specific serving framework. A live playground is at aka.ms/vibeasr, and the source repository is at github.com/microsoft/VibeVoice. The "7B" in the model name is a parameter hint derived from the name itself; the model card does not state an explicit parameter count. No hardware minimum, per-frame latency, or per-language accuracy figure appears in the card text.

Our read

The interesting part is not the ten-language list. It is that speaker attribution and streaming transcription share a single forward pass. The old workflow for a multi-speaker explainer or interview video was: run ASR, run diarisation, align the two, and hope the timestamps survived. This output already carries a speaker label per segment as audio arrives. That collapses a two-model stack into one and removes the alignment step.

The hotword mechanism is the second practical detail. If a client's product is called "Zylo" and the base model has never seen that token, a hotword entry tells the model what to listen for. Small thing, but it saves re-transcription passes on niche content.

What the model card does not give you is the part a studio needs before wiring this into a production graph: minimum VRAM, per-frame latency, concurrent-stream limit, supported sample rate, maximum clip length. None are stated. The endpoints_compatible tag is a hint, not a spec. The only way to confirm real-time performance on a 16 GB consumer GPU is to run it and measure.

The MIT licence is quietly the most important line for a commercial studio: no per-use fee, no non-commercial restriction, no deployment ceiling. Embed it in a client deliverable or a SaaS pipeline and ship.

What this changes

On Monday, nothing is turnkey. The model is a transformers pipeline with safetensors weights; there is no ready-made ComfyUI node in the brief. You would write a thin Python wrapper that calls the streaming pipeline and emits speaker-tagged segments, then bridge that into a ComfyUI graph or a subtitle burn-in pass. The hotword list is the first thing to populate with client-specific product names and character names. Ten languages cover most EU and CJK markets a small studio serves; Scandinavian, Arabic, and Hindi are absent. Before committing, run a proof-of-concept on your target GPU and log per-frame latency, because no benchmark is published.

License

MIT. Commercial use is permitted without restriction: no per-use fee, no non-commercial clause, no model-size or revenue ceiling. The only obligation is to include the MIT licence text with the distribution. For a studio shipping client work or a product built on this model, that is the permissive end of the open-weights spectrum.

Key takeaways

• VibeVoice-ASR-Streaming-7B is a streaming, speaker-attributed ASR model from Microsoft Research, published 2 September 2026, covering ten languages and accepting user-defined hotwords for domain-specific vocabulary.
• Speaker attribution is built into the transcription pass, removing the need for a separate diarisation model and the alignment step between the two.
• No hardware minimum, per-frame latency, or per-language accuracy figure is stated in the model card; a proof-of-concept on your target GPU is the only way to confirm real-time viability.
• MIT licence: commercial use, embedding in products, and client delivery are all permitted with no restriction beyond the licence text.
• Integration with ComfyUI is not turnkey; a custom transformers-based node or an external API bridge is required.

Sources

1. microsoft/VibeVoice-ASR-Streaming-7B — automatic-speech-recognition on Hugging Face — tier 2