Ar4ikov / VibeVoice-ASR-Streaming-7B-DFlash2

VibeVoice-ASR-Streaming-7B-DFlash2

VibeVoice-ASR-Streaming-7B , the original BF16 weights as Microsoft published them, with its DFlash 2 drafter bundled in drafter/ : one download, and vibevoice.c decodes with speculative decoding -- the drafter proposes 8 tokens in one pass, the model checks them in one pass and keeps the ones it agrees with. The check is exact : every checked row is computed with the arithmetic of the model's own decode step, so the transcript is byte-for-byte the one without the drafter.

Use

Needs vibevoice.c with DFlash 2 support: branch dflash2 ( PR #48 ), in the next release. A model directory's drafter/ is used without asking:

vv_cli --model ./VibeVoice-ASR-Streaming-7B-DFlash2 --audio talk.wav                  # with the drafter
vv_cli --model ./VibeVoice-ASR-Streaming-7B-DFlash2 --audio talk.wav --draft none     # plain decoding
vv_cli serve --model ./VibeVoice-ASR-Streaming-7B-DFlash2 --slots 4                  # streaming sessions (WebSocket, SSE) too

vibevoice.c runs BF16 weights as dense FP16 (about 19 GB of VRAM for a 7B, drafter included); --quant int4 quantizes them at load, and the drafter works with that too.

Results

vibevoice.c b72be15 (branch dflash2 ), RTX 3090, greedy decoding, decode tokens per second:

BF16 weights run as dense FP16; streaming sessions (22 + 4 frames a chunk). Plain = the same model with --draft none ; --draft-check exact .

Inside

• The model: the files of microsoft/VibeVoice-ASR-Streaming-7B at revision 60d858b5 , unchanged (17.36 GB) -- its card has the model, the evaluation and the license.
• drafter/ : Ar4ikov/VibeVoice-ASR-Streaming-7B-DFlash2-Drafter at revision 1d45ecb5 (1.66 GB): 5 Qwen3-style layers reading the model's layers 1/7/13/19/25, a candidate selector, a 32768-id draft vocabulary; BF16, held as INT4 by default at load ( --draft-quant f16 keeps FP16). Its card has the architecture and the training.

License

MIT, like VibeVoice.