ayousanz / pocket-tts-ja-experiments

Pocket TTS Japanese — ended experiments archive

This is an archive of unfinished Japanese Pocket TTS experiments, not a finished Japanese TTS release. Development ended on 2026-10-15 at the author's request after takuma104/pocket-tts-ja became available. No additional training or quality selection is planned. The author subsequently authorized preserving the existing local models in this public repository with manual access approval .

日本語Pocket TTSの実験成果物を保存するためのリポジトリです。開発は終了しています。実用品質の合格モデルではありません。モデルページは公開し、ファイル取得は手動承認のgateとしています。旧 Phase 1検証モデル とは別の保存先で、Phase 1は変更していません。

Contents

Upload status: incomplete, waiting for the public-storage quota refresh. The student5k EMA export and teacher2k/10k checkpoints are present. Teacher5k/20k checkpoints and the legacy220 export are still pending. ARTIFACTS.json describes the intended final inventory; presence must be checked against the actual file list while the upload is incomplete.

Raw corpus audio, evaluation voice-prompt recordings, audio-codec latents and cloud credentials are not distributed here. The upstream English initializer is not duplicated. The 5,005-step technical resume-probe weights were not recovered and are not presented as another quality candidate. Historical recovery archives containing audio references are not uploaded.

The initial upload including optimizer sidecars reached the account's public storage limit. This repository therefore prioritizes the models: optimizer sidecars, the student's duplicate raw/EMA training checkpoint and the legacy duplicate training checkpoint remain in the author's local archive. The existing model weights are transferred without conversion or new inference. Previous recovery proofs describe the wider local archive; their presence here does not mean all files named by a recovery proof are distributed on HF.

Training provenance

The GOL M2a teacher starts from the released English 24-layer Pocket TTS model, with a reinitialized Japanese text embedding and SP8000. The training audio for this run is GOL only : 723.507 hours / 447,090 rows for training and 22.329 hours / 12,951 rows for validation, after alignment, filtering, synchronized long-clip splitting and exact PCM/ID-group split checks. These label groups are not verified identities of individual actors.

The tokenizer predates this run and was trained using GOL + MoeSpeech text. “GOL only” refers to the current M2a and student training audio, not to the tokenizer's entire history. The separate legacy 220-step run used an earlier small Japanese validation corpus; it is not the GOL-only experiment and was not used to initialize it.

The 6-layer student starts from teacher layers 0, 1, 2, 21, 22 and 23 of the provisional 10k EMA teacher, using the same SP8000 and Mimi. Guidance scale 2 is distilled into the student. It was trained for 5,000 updates at learning rate 4e-4 with 1,000 warmup steps, effective batch size 64 and EMA decay 0.999. The earlier 35-step technical pilot was not its initializer.

Results and limitations

Full teacher/student evaluation covers 3,024 conditions: 20 held-out ID-group prompts plus one training prompt, 12 texts and 12 generation profiles. The primary comparison uses 20 held-out prompts × 12 texts, trainer EMA, EOS threshold -4, teacher CFG2 and student CFG1 (guidance distilled into the student).

Teacher 20k has primary-condition CER 15.01%, worse than 10k in that profile, although other profiles improve. No final teacher was selected. The 5k student has substantial reading errors and is not accepted for production. Removing EMA does not close the gap (raw student CER 42.37% in the same CFG1/EOS-4 profile).

CPU FP32/INT8 evaluation was performed on Vast.ai Linux, using one fixed voice prompt, 12 texts, two passes and four model/precision combinations (96 requests). RTF medians were teacher FP32 0.521, teacher INT8 0.183, student FP32 0.156 and student INT8 0.069. These are not same-quality speed comparisons: guidance differs, the student misreads, some requests hit generation limits, and RTF uses generated duration. They do not demonstrate Windows performance or broad voice coverage.

Human listening covers only one normal-sentence pair ( held_00 / daily ). The teacher's spoken content matched the intended sentence. The user confirmed that the student actually said こんにちは。今日はいい天気っすね。きっ ; this is not an ASR-only error. Long-form listening, naturalness, ending-position review and final quality selection remain incomplete.

The archived student export was loaded on Vast.ai without a checkpoint, with frozen source and an empty offline HF cache; one sentence matched checkpoint-path PCM. That establishes export/load equivalence for that sample, not quality. Restoring raw/EMA/optimizer and continuing 5,000→5,005 updates was also tested. RNG and dataloader position are not checkpointed, so bitwise equivalence to uninterrupted training is not claimed.

Formats and reuse

Teacher .pt files contain training checkpoints (including raw weights and EMA), not standalone safetensors exports. For student5k, model.safetensors is the separately verified merged EMA export. student5000.yaml at the repository root is the archived inference configuration with the two artifact locations changed to this repository's HF URLs. The original relative-path configuration is retained under gol/student/5000/model.yaml .

Reuse the frozen Japanese source in source/stage4-source.tar and its recorded configuration. Compatibility with a different/current stock Pocket TTS wheel has not been newly tested. No model generation, ASR or training was performed during this archival upload.

After approval, files can be retrieved using an authenticated Hugging Face client. Access approval does not change the quality limitations or the license. Source audio and computed latents on the discontinued Vast.ai instance were not retained externally; reconstructing training data would require retrieval and preprocessing again.

License and attribution

Model weights inherit CC BY 4.0 from Kyutai Pocket TTS . These artifacts modify its tokenizer/text embedding and Japanese language-model weights; the student additionally reduces FlowLM from 24 to 6 layers and distills CFG2. This experiment does not fine-tune Mimi. Source code is MIT; the repository's LICENSE refers to source code. The weights' license is CC BY 4.0 .

Training audio is not included or relicensed. See the original GOL dataset and previous Phase 1 provenance for historical context. No claim of independence from all source-data rights is made.

Thanks to Kyutai for Pocket TTS, its English weights, training code and Mimi-based pipeline. These experiments are preserved for reproducibility, including their failure cases and incomplete human review.