Audio8 / Audio8-TTS-Preview-0.1b

Audio8 TTS 0.1B supports speech generation and zero-shot voice cloning. This repository contains the complete v4 mixed checkpoint, its neural audio codec, tokenizer, processor, and Hugging Face remote code.

Compact Scale

The defining characteristic of this release is its size. The main generative model is approximately 170M parameters , while the codec decoder is a separate approximately 120M-parameter component. Even counting the codec decoder, the complete audio generation stack remains much smaller than most modern multilingual TTS systems.

These figures are approximate reference scales collected from the respective model reports and are not a strictly matched parameter-count audit. The 0.1B checkpoint is intended to make zero-shot TTS practical with a much smaller language/audio model footprint, not to claim identical quality across every language or benchmark.

Supported Languages

• Primary: Chinese and English
• Experimental/multilingual evaluation: German, Spanish, French, Italian, Japanese, and Korean

Model Details

The model uses an Audio8 Falcon H1 architecture with slow and fast autoregressive branches. The slow branch predicts semantic tokens, while the fast branch predicts codec codebooks conditioned on the slow hidden state.

The codec is included in this repository. No additional codec checkpoint is required.

Installation

Python 3.11 or newer and a CUDA-capable GPU are recommended.

pip install "torch>=2.5.0" "torchaudio>=2.5.0" \
  "transformers>=4.57.0,<5" "soundfile>=0.12" "safetensors>=0.4"

Usage

The model includes custom Transformers code. Load it with trust_remote_code=True .

Voice cloning

The primary usage of this checkpoint is zero-shot voice cloning. Replace reference.wav and the reference transcript with your own audio and text. The reference transcript should match the spoken content of the reference audio.

import soundfile as sf
import torch
from transformers import AutoModel, AutoProcessor

model_id = "Audio8/Audio8-TTS-Preview-0.1b"
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.bfloat16 if device == "cuda" else torch.float32

processor = AutoProcessor.from_pretrained(
    model_id,
    trust_remote_code=True,
)
model = AutoModel.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype=dtype,
).eval().to(device)

inputs = processor(
    text=["这是一个语音合成测试。"],
    reference_audio=["reference.wav"],
    reference_text=["参考音频对应的完整文本。"],
    return_tensors="pt",
)
inputs = {name: value.to(device) for name, value in inputs.items()}

with torch.inference_mode():
    output = model.generate(
        **inputs,
        max_new_tokens=512,
        temperature=0.7,
        top_p=0.9,
        top_k=50,
        do_sample=True,
        return_dict_in_generate=True,
    )
    waveforms, waveform_lengths = model.decode_audio(output.codes)

audio = waveforms[0, : int(waveform_lengths[0])].float().cpu().numpy()
sf.write("output.wav", audio, model.config.codec_sample_rate)

For synthesis without cloning, omit reference_audio and reference_text . For batch inference with audio or pre-encoded reference codes, see the Audio8 TTS training and inference repository.

Evaluation

Lower WER/CER is better; higher SIM (similarity) is better.

CV3 error-rate comparison

Lower is better. These comparison values follow the evaluation table published for Audio8 TTS Preview 0.6B; they are reference comparisons rather than a strictly matched re-evaluation.

Seed-TTS comparison

Similarity values are shown as percentages in this comparison table. Lower WER/CER is better; higher similarity is better.

The IndexTTS2.5 row uses the Token-Level Concatenation result from the Seed-TTS-Eval portion of Table 1 in the IndexTTS 2.5 technical report.

Parameter scales are approximate reference values from the respective model reports (see the Compact Scale section); they are not a strictly matched parameter-count audit. For reference, MOSS-TTS contains 8,489,841,664 parameters and VoxCPM2's main model contains 2,290,004,544 parameters; the separate AudioVAE is not included in the parameter comparison.

Fish S2 Pro was reevaluated because its official evaluation uses its own normalizer. Higgs Audio v2 was evaluated locally because concrete values were unavailable. All other baseline values were collected from their official reports through the VoxCPM repository .

Different normalizers and evaluators make cross-project values reference comparisons rather than a strictly matched ranking. Evaluation coverage does not expand the Preview checkpoint's supported-language claim beyond the languages listed above.

Limitations and Responsible Use

• This is a compact preview checkpoint. Chinese and English are the primary target languages; other languages generally show weaker and more variable quality.
• Very long, noisy, or incorrectly transcribed reference clips can reduce generation stability and speaker similarity.
• Generated speech can be misused for impersonation or misinformation. Obtain consent before cloning a voice and disclose synthetic audio where appropriate.
• Evaluate the model for accuracy, safety, and legal compliance before deployment.

License

This model repository is released under the Creative Commons Attribution-NonCommercial 4.0 International License .