FluidInference / fsmn-vad-coreml

FSMN-VAD — CoreML (Apple Neural Engine)

CoreML conversion of FunASR's FSMN-VAD (~5.2M params), for on-device voice activity detection on Apple Silicon. Upstream: iic/speech_fsmn_vad_zh-cn-16k-common-pytorch .

Files

Pipeline

waveform → [Preprocessor fp32/CPU] → features [1,T,400]
        → [FSMN fp16/ANE] → scores [1,T,248]
        → host: silence_prob = softmax(scores)[:, sil_pdf_ids].sum()  (sil_pdf_ids=[0])
        → state machine (thresholds in vad_config) → speech segments [start_ms, end_ms]

• Frame rate: 10 ms (LFR n=1, no downsampling).
• The segment decision logic (FunASR FsmnVADStreaming ) runs on the host: silence/speech hysteresis with max_end_silence_time (800 ms), max_start_silence_time (3000 ms), max_single_segment_time (60 s), sil_to_speech_time_thres (150 ms). See vad_config.json .

Benchmark — fidelity vs FunASR (FLEURS zh, n=50)

Parity: preprocessor matches WavFrontendOnline max|Δ|≈3e-5; FSMN scorer max|Δ| 0.0016. Boundaries match FunASR within ~50 ms.

License

Weights derive from FunASR's FSMN-VAD; upstream license applies. Format conversion only.