yang-qi1222 / cosyvoice3_stream.cpp

cosyvoice3_stream.cpp

Runtime assets for cosyvoice3_yq_cpp , an inference-only CosyVoice3 C++/GGML streaming runtime.

This repository contains one project-compressed GGUF model, the two ONNX models needed to extract a voice prompt from reference audio, and ten small precomputed prompt features for immediate testing. Source code is distributed separately on Gitee.

Files

Every binary is pinned in SHA256SUMS . The source repository also ships a machine-readable manifest used by its downloader.

Model Lineage

The base model is FunAudioLLM/Fun-CosyVoice3-0.5B-2512 . The published GGUF applies the following project-specific deployment changes:

• a 12-layer student speech LLM obtained through structured layer pruning and distillation;
• Flow inference distilled from 10 steps to 4 steps;
• GGUF conversion with inner LLM MLP tensors quantized to Q8 and Flow/HiFT kept in F16.

The LLM compression workflow was informed by SPADE , with additional material on the SPADE project page . This is an independent CosyVoice3 engineering adaptation, not an official SPADE model or a claim that the paper's reported metrics transfer to this model.

Prompt Features

control10_01 through control10_10 are features extracted from synthetic teacher-model outputs used in this project's five-male/five-female Control10 evaluation. No source WAV files are distributed, and these prompts are not presented as the voice of any real person.

Use a prompt directly for the lowest startup overhead. To clone a new authorized reference voice, use speech_tokenizer_v3.onnx and campplus.onnx with the audio-enabled runtime to create a new prompt_speech.gguf .

Download And Run

git clone https://gitee.com/yang-qi1222/cosyvoice3_yq_cpp.git
cd cosyvoice3_yq_cpp
python3 -m pip install -r requirements-tools.txt
python3 scripts/download_assets.py \
  --manifest manifests/assets.example.json \
  --asset-root assets

Build and start the feature-only CPU server:

scripts/build_runtime.sh \
  --mode feature \
  --backend cpu \
  --build-dir build/feature-cpu

scripts/start_server.sh \
  --build-dir build/feature-cpu \
  --backend cpu \
  --model assets/models/student12_mlp_inner_q8_flow4_hift_f16.gguf \
  --prompt-speech assets/prompts/control10_01.gguf \
  --voice control10_01 \
  --port 8080

CUDA architecture, thread count, chunk size, and FlashAttention settings must be selected and validated for the deployment machine. See the source repository for the audio-input build and complete server examples.

Measured Baseline

The accepted project baseline covers only an NVIDIA RTX 5880 Ada Generation GPU, one serial request at a time, 16 host threads, chunk_tokens=75 , LLM FlashAttention disabled, and Flow FlashAttention enabled.

These measurements must not be extrapolated to RTX 4060, Jetson AGX, other GPU architectures, concurrent service, or long-context synthesis.

Limitations

• This repository distributes inference assets, not training checkpoints or training code.
• The reference-audio path also requires an ONNX Runtime C/C++ SDK at build time; the two ONNX model files do not replace that dependency.
• Voice cloning must only be performed with appropriate speaker consent and data rights.
• The project is community maintained and is not affiliated with or endorsed by the official CosyVoice or SPADE teams.

License And Acknowledgements

Model assets are distributed under Apache-2.0. The C++ source repository uses the MIT license; source and model licenses are separate.

This work depends on and thanks:

• QwenAudio/CosyVoice and the CosyVoice3 paper ;
• Lourdle/cosyvoice.cpp ;
• SPADE ;
• ggml , llama.cpp , and ONNX Runtime .

Please cite the original CosyVoice3 and SPADE papers when this model is used in research comparisons.