FluidInference / silero-vad-v2-coreml

Silero VAD v2 CoreML

Enhanced Silero VAD v3 model with trained Squeeze-Excitation (SE) modules for improved voice activity detection on Apple platforms.

Model Description

This is an enhanced version of the Silero VAD v3 model, converted to CoreML format with additional SE modules trained on the MUSAN dataset. The model provides state-of-the-art voice activity detection performance optimized for Apple Silicon.

Key Features

• Single unified model replacing the previous 3-model pipeline
• Trained SE modules for better noise/music suppression
• 92% accuracy on MUSAN test set (F1-score: 91.3%)
• stateless we will add a stateful version in future uploads for 10%+ improvements

Architecture

The model uses the Silero VAD v3 architecture with the following enhancements:

• LSTM-based encoder with 4 blocks
• Squeeze-Excitation modules in each encoder block (trained on MUSAN)
• LayerNorm after LSTM with 0.15 scaling factor
• Input: 512 audio samples at 16kHz (32ms chunks)
• Output: Voice probability [0, 1]

Performance

Based on 100 files, 50 noise, 50 speech

Files

• silero_vad.mlmodelc - Compiled CoreML model (for production)

Usage

Swift Integration

import CoreML

// Load the model
let modelURL = Bundle.main.url(forResource: "silero_vad", withExtension: "mlmodelc")!
let model = try MLModel(contentsOf: modelURL)

// Prepare input (512 samples at 16kHz)
let audioArray = try MLMultiArray(shape: [1, 512], dataType: .float32)
// ... fill audioArray with normalized audio samples [-1, 1]

// Run inference
let input = try MLDictionaryFeatureProvider(dictionary: ["audio_chunk": audioArray])
let output = model.prediction(from: input)
let probability = output.featureValue(for: "vad_probability")!.multiArrayValue![0].floatValue

// Apply threshold
let isVoiceActive = probability >= 0.5

Important Notes

1. Audio Normalization : Input audio must be normalized to [-1, 1] range
2. Chunk Size : Fixed at 512 samples (32ms at 16kHz)
3. Threshold : Recommended threshold is 0.5 (adjustable based on use case)

Training Details

The SE modules were trained on the MUSAN dataset with:

• 86.47% validation accuracy on MUSAN
• Batch size: 32
• Learning rate: 1e-3
• Training samples: ~45,000 chunks
• Optimizer: Adam

Requirements

• macOS 13.0+ / iOS 16.0+
• CoreML framework
• Apple Silicon recommended for optimal performance

License

Model weights derived from Silero VAD v3 (MIT License). SE module enhancements and CoreML conversion by FluidInference.

Citation

If you use this model, please cite:

@misc{silero_vad_v2_coreml,
  title={Silero VAD v2 CoreML},
  author={FluidInference},
  year={2024},
  publisher={Hugging Face},
  url={https://huggingface.co/FluidInference/silero-vad-v2-coreml}
}

Acknowledgments

• Original Silero VAD by Silero Team
• MUSAN dataset for training
• CoreML framework by Apple