Step-Audio-TTS-3B represents the industry's first Text-to-Speech (TTS) model trained on a large-scale synthetic dataset utilizing the LLM-Chat paradigm. It has achieved SOTA Character Error Rate (CER) results on the SEED TTS Eval benchmark. The model supports multiple languages, a variety of emotional expressions, and diverse voice style controls. Notably, Step-Audio-TTS-3B is also the first TTS model in the industry capable of generating RAP and Humming, marking a significant advancement in the field of speech synthesis.
This repository provides the model weights for StepAudio-TTS-3B, which is a dual-codebook trained LLM (Large Language Model) for text-to-speech synthesis. Additionally, it includes a vocoder trained using the dual-codebook approach, as well as a specialized vocoder specifically optimized for humming generation. These resources collectively enable high-quality speech synthesis and humming capabilities, leveraging the advanced dual-codebook training methodology.
Performance comparison of content consistency (CER/WER) between GLM-4-Voice and MinMo.
Model
test-zh
test-en
CER (%) ↓
WER (%) ↓
GLM-4-Voice
2.19
2.91
MinMo
2.48
2.90
Step-Audio
1.53
2.71
Results of TTS Models on SEED Test Sets.
StepAudio-TTS-3B-Single denotes dual-codebook backbone with single-codebook vocoder*
Model
test-zh
test-en
CER (%) ↓
SS ↑
WER (%) ↓
SS ↑
FireRedTTS
1.51
0.630
3.82
0.460
MaskGCT
2.27
0.774
2.62
0.774
CosyVoice
3.63
0.775
4.29
0.699
CosyVoice 2
1.45
0.806
2.57
0.736
CosyVoice 2-S
1.45
0.812
2.38
0.743
Step-Audio-TTS-3B-Single
1.37
0.802
2.52
0.704
Step-Audio-TTS-3B
1.31
0.733
2.31
0.660
Step-Audio-TTS
1.17
0.73
2.0
0.660
Performance comparison of Dual-codebook Resynthesis with Cosyvoice.
Token
test-zh
test-en
CER (%) ↓
SS ↑
WER (%) ↓
SS ↑
Groundtruth
0.972
-
2.156
-
CosyVoice
2.857
0.849
4.519
0.807
Step-Audio-TTS-3B
2.192
0.784
3.585
0.742
More information
For more information, please refer to our repository:
Step-Audio
.
Runs of stepfun-ai Step-Audio-TTS-3B on huggingface.co
149
Total runs
-1
24-hour runs
3
3-day runs
-6
7-day runs
-143
30-day runs
More Information About Step-Audio-TTS-3B huggingface.co Model
Step-Audio-TTS-3B huggingface.co is an AI model on huggingface.co that provides Step-Audio-TTS-3B's model effect (), which can be used instantly with this stepfun-ai Step-Audio-TTS-3B model. huggingface.co supports a free trial of the Step-Audio-TTS-3B model, and also provides paid use of the Step-Audio-TTS-3B. Support call Step-Audio-TTS-3B model through api, including Node.js, Python, http.
Step-Audio-TTS-3B huggingface.co is an online trial and call api platform, which integrates Step-Audio-TTS-3B's modeling effects, including api services, and provides a free online trial of Step-Audio-TTS-3B, you can try Step-Audio-TTS-3B online for free by clicking the link below.
stepfun-ai Step-Audio-TTS-3B online free url in huggingface.co:
Step-Audio-TTS-3B is an open source model from GitHub that offers a free installation service, and any user can find Step-Audio-TTS-3B on GitHub to install. At the same time, huggingface.co provides the effect of Step-Audio-TTS-3B install, users can directly use Step-Audio-TTS-3B installed effect in huggingface.co for debugging and trial. It also supports api for free installation.