acul3 / Qwen3-TTS-1.7B-Base-ExecuTorch

Qwen3-TTS Voice Clone — ExecuTorch (Android-ready)

On-device text-to-speech with voice cloning, converted from Qwen3-TTS-1.7B-Base to ExecuTorch .pte format for mobile/edge deployment.

1.9B parameter end-to-end TTS — clone any voice from a short audio sample and synthesize speech entirely on-device. No cloud, no internet needed.

Models

INT8 Quantized (⭐ Recommended for on-device)

FP32 Unquantized

Auxiliary Files

Architecture

Qwen3-TTS Voice Clone Pipeline (1.9B params total)

Input: text + reference audio (3-5s voice sample)
                    │
        ┌───────────┴───────────┐
        ▼                       ▼
  Speaker Encoder (12M)    Speech Tokenizer
  TDNN → AttPool → FC     (encode ref audio
  ref_audio → x_vector     to codec codes)
  [1, 2048]                [T, 16]
        │                       │
        └───────┬───────────────┘
                ▼
         Talker LM (1.7B)
         Qwen3, 28 layers, GQA 16/8
         dim=2048, audio vocab=3072
         Autoregressive codec generation
                │
                ▼
         Code Predictor (175M)
         Predict 15 additional codebooks
         per token (residual VQ)
                │
                ▼
         Vocoder (154M)
         Codec tokens → 24kHz PCM audio
                │
                ▼
         Output: speech waveform (.wav)

Component Details

How It Works

Voice Clone Pipeline

1. ref_audio (24kHz, 3-5s) → mel_spectrogram → speaker_encoder.pte → x_vector [1, 2048]
2. ref_audio → speech_tokenizer.encode() → ref_codes [T, 16]  (runs on CPU, not exported)
3. text → Qwen2 tokenizer → input_ids
4. Embed: text_embedding(input_ids) + codec_embedding(ref_codes) + x_vector → inputs_embeds
5. talker.pte(inputs_embeds, kv_cache, ...) → codec logits  (autoregressive loop)
6. code_predictor.pte(hidden_states) → codebooks 2-16  (per step)
7. All codec codes → vocoder.pte → 24kHz PCM waveform

Token Format

The talker uses an interleaved text+audio token sequence:

[BOS] [text tokens...] [speaker x-vector] [ref audio codes...] [SEP] [generated audio codes...]

Key Parameters

Quick Start — Python

from huggingface_hub import hf_hub_download
from executorch.runtime import Runtime
import torch
import numpy as np

REPO = "acul3/Qwen3-TTS-1.7B-Base-ExecuTorch"

# Download INT8 models
spk_path = hf_hub_download(REPO, "speaker_encoder_int8.pte")
talker_path = hf_hub_download(REPO, "talker_int8.pte")
cp_path = hf_hub_download(REPO, "code_predictor_int8.pte")
voc_path = hf_hub_download(REPO, "vocoder_int8.pte")
emb_path = hf_hub_download(REPO, "talker_embeddings.pt")
cp_extras_path = hf_hub_download(REPO, "code_predictor_extras.pt")

# Load ExecuTorch runtime
runtime = Runtime.get()
speaker_enc = runtime.load_program(spk_path).load_method("forward")
vocoder = runtime.load_program(voc_path).load_method("forward")

# Load embeddings (used in Python orchestration)
embeddings = torch.load(emb_path, weights_only=True)
cp_extras = torch.load(cp_extras_path, weights_only=True)

# For full generation, see scripts/test_e2e.py

Quick Start — Android (Kotlin)

import org.pytorch.executorch.Module

// Load all 4 modules
val speakerEnc = Module.load("speaker_encoder_int8.pte")
val talker = Module.load("talker_int8.pte")
val codePred = Module.load("code_predictor_int8.pte")
val vocoder = Module.load("vocoder_int8.pte")

// Pipeline:
// 1. Process ref audio → mel → speaker_enc.forward() → x_vector
// 2. Build input embeddings (text + speaker + ref codes)
// 3. Autoregressive loop: talker.forward() → codec token → code_pred.forward() → all codebooks
// 4. vocoder.forward(all_codes) → PCM audio
// See scripts/ for full implementation details

Validation Results

*Speaker encoder: 0.965 due to mel padding for fixed-size export. With matching sizes: 1.000.

INT8 quantization produces valid, intelligible speech — tested with full pipeline generation.

Export Details

Export Challenges Solved

1. Conv1d padding="same" — Replaced with explicit F.pad() + Conv1d(padding=0) (ExecuTorch doesn't support padding="same")
2. DynamicCache — Replaced with static KV cache tensors as model inputs/outputs
3. MROPE (Multi-Resolution RoPE) — Simplified: all 3 dimensions share identical position_ids for TTS
4. Stride-0 tensors — Used .repeat() instead of .expand() for ExecuTorch compatibility
5. Vocoder dynamic chunking — Bypassed chunked_decode with fixed code length

Scripts

Hardware Requirements

On-device inference (INT8)

• RAM : 8 GB minimum (models use ~1.8 GB + KV cache + audio buffers)
• Storage : 1.8 GB for all 4 model files + extras
• CPU : ARM64 (Android) or x86_64

Export/development

• 64 GB+ unified memory recommended (Jetson AGX Orin or similar)
• Python 3.10+, PyTorch 2.10+, ExecuTorch 1.1.0, torchao

Reproduce

# Setup
git clone https://huggingface.co/acul3/Qwen3-TTS-1.7B-Base-ExecuTorch
cd Qwen3-TTS-1.7B-Base-ExecuTorch
pip install executorch torchao qwen-tts

# Export all modules
python scripts/export_speaker_encoder.py
python scripts/export_talker.py
python scripts/export_code_predictor.py
python scripts/export_vocoder.py

# Quantize
python scripts/quantize_all.py

# Validate
python scripts/test_e2e.py

License

Apache 2.0 (same as source model )

Citation

@misc{qwen3tts_executorch_2026,
  title = {Qwen3-TTS-1.7B-Base-ExecuTorch: On-Device Voice Clone TTS},
  author = {Samsul Rahmadani},
  year = {2026},
  url = {https://huggingface.co/acul3/Qwen3-TTS-1.7B-Base-ExecuTorch},
  note = {Converted from Qwen/Qwen3-TTS-1.7B-Base}
}

Acknowledgments

• Qwen Team for the Qwen3-TTS model
• PyTorch ExecuTorch for the on-device runtime
• torchao for INT8 weight-only quantization