aufklarer / CosyVoice3-0.5B-MLX-4bit

CosyVoice3-0.5B MLX 4-bit

CosyVoice 3 text-to-speech model converted to MLX safetensors format with 4-bit quantization for Apple Silicon inference.

Converted from FunAudioLLM/Fun-CosyVoice3-0.5B-2512 .

Model Details

Pipeline

Text → LLM (Qwen2.5-0.5B) → Speech Tokens (FSQ 6561) → DiT Flow Matching → Mel (80-band) → HiFi-GAN → Audio (24kHz)

Languages

Chinese, English, Japanese, Korean, German, Spanish, French, Italian, Russian

Files

• llm.safetensors — LLM weights (4-bit quantized)
• flow.safetensors — DiT flow matching decoder (fp16)
• hifigan.safetensors — HiFi-GAN vocoder (fp16, weight-norm folded)
• config.json — Model configuration

Conversion Details

• LLM: 4-bit quantization (group_size=64) of attention projections, MLP, and speech head
• Flow: fp16 (flow matching is sensitive to quantization)
• HiFi-GAN: fp16 with weight normalization folded ( w = g * v / ||v|| )
• Conv1d weights transposed from PyTorch [out, in, kernel] to MLX [out, kernel, in]

Usage

For use with Speech Swift :

import CosyVoiceTTS

let model = try await CosyVoiceTTSModel.fromPretrained()
let audio = model.synthesize(text: "Hello, how are you?", language: "english")

License

Apache 2.0 (same as upstream CosyVoice 3)

Citation

@article{du2025cosyvoice3,
  title={CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-training},
  author={Du, Zhihao and others},
  journal={arXiv preprint arXiv:2505.17589},
  year={2025}
}