litert-community / Kokoro-82M

Kokoro-82M — LiteRT (free-text, bucketed)

⚠️ Labeled preview — FP32, CPU. Arbitrary free text → speech (not a single baked sentence). The neural graphs are LiteRT .tflite ; two small steps run host-side (the hn-NSF source STFT and the final iSTFT overlap-add). GPU + quantization are the next steps.

A LiteRT ( .tflite ) conversion of hexgrad/Kokoro-82M (StyleTTS2 + ISTFTNet) for on-device free-text text-to-speech (arbitrary text in, 24 kHz speech out). Audio fidelity ≈ 0.9994 magnitude-spectrogram correlation to the PyTorch reference (verified across multiple held-out sentences, not just the export sample).

Kokoro has one data-dependent length (the duration→alignment expansion L = sum(pred_dur) ), which litert_torch cannot keep dynamic (the LSTM sequence axis specializes). It is therefore split into three fixed-bucket bundles with host steps between them, so arbitrary text works by left-padding to the bucket and trimming the output (longer text is split into sentences host-side, each ≤ the bucket):

text --(G2P, host)--> phoneme ids
1. kokoro_predictor.tflite : ids[1,128], ref_s[1,256], attn[1,128] -> duration, d, t_en
   host: pred_dur = round(duration); alignment one-hot aln[1,128,512]; frame_mask[1,512]
2. kokoro_prosody.tflite   : d, t_en, aln, ref_s, frame_mask       -> asr, F0, N
   host: har = STFT(SineGen(f0_upsamp(F0)))     (the hn-NSF excitation)
3. kokoro_vocoder.tflite   : asr, F0, N, har, ref_s, frame_mask    -> spec, phase
   host: iSTFT overlap-add(spec, phase) -> 24 kHz waveform; trim to L*600

Files

Token bucket T = 128 , frame bucket L = 512 (≈ 12.8 s of audio per chunk at 24 kHz). Bundles are voice-independent — the voice is the ref_s input (a voices/*.pt from the base repo, indexed by token-sequence length).

Fixed-length demo build (litert-samples)

kokoro_82m_fixedlen_fp32.tflite is the earlier single-graph build (fixed-length input, baked demo sentence length, spec+phase out, host-side iSTFT) kept for the official litert-samples text_to_speech sample , which downloads it by this name. Measured on Pixel 8a (fp32, CPU, 4 threads): RTF ≈ 1.8 (~6.6 s to synthesize 3.7 s of audio) — quantization is the path to realtime on-device.

Specs

How it was converted

• Stock official converter ( litert_torch ), general path — Kokoro is StyleTTS2/ISTFTNet, not a transformer LLM, so the Generative-API re-authoring path does not apply.
• Three fixed-bucket bundles because the dynamic alignment length can't stay symbolic through the converter (the dynamic-LSTM wall). Every workaround is load-bearing and numerically faithful: the 6 bidirectional LSTMs are unrolled as masked bi-LSTMs that carry state through right-padding (a fused nn.LSTM leaks pad tokens into the backward pass and wrecks prosody); the 58 AdaIN InstanceNorms normalize over real frames only so bucket pad frames don't poison the statistics; the hn-NSF source STFT runs host-side (its atan2 phase flips at the F0→0 pad boundary on-device).
• iSTFT runs host-side (vocoder emits spec+phase): the in-graph conv-transpose iSTFT hits a converter weight-dedup bug that fuses the cos/sin DFT bases. The host overlap-add (no learned weights) is numerically exact.

Training data

Inherited from hexgrad/Kokoro-82M : a few hundred hours of permissive / non-copyrighted audio — public-domain audio, audio under permissive licenses (e.g. Koniwa tnc CC BY 3.0, SIWIS CC BY 4.0), and synthetic audio from large-provider TTS — paired with IPA phoneme labels. No proprietary or scraped personal recordings, no custom voice clones. This LiteRT artifact is a format conversion of the released weights and introduces no additional training data.

PII

No personally identifiable information is included. Per the base model's disclosure the training audio is permissive / public-domain / synthetic rather than scraped personal recordings; to the best of our knowledge the released weights contain no PII, and the conversion adds none.

Front-end (free-text G2P)

For arbitrary text with no dropped words (names, brands, numbers), pair with the neural grapheme-to-phoneme front-end: litert-community/Kokoro-G2P-en-US .

Sample / code

• Official Android sample (Interpreter API, fixed-length demo build): google-ai-edge/litert-samples #159
• Conversion / pipeline code: https://github.com/john-rocky/LiteRT-Models

Roadmap

• GPU : the attention's fused-QKV >4-D layout + mask EQUAL / SELECT keep it on CPU; decomposing attention to ≤4-D would let it ride the GPU delegate. In-graph iSTFT on GPU additionally needs an FFT kernel in the LiteRT GPU delegate (ML Drift).
• Quantization (int8/int4) is the obvious next step.

Status

Labeled preview — the converted model is parity-verified. The official litert-samples text_to_speech sample (fixed-length demo build) is in review at google-ai-edge/litert-samples #159 ; a free-text sample (3-bundle + G2P front-end) is in progress.

License

Apache-2.0, inherited from hexgrad/Kokoro-82M .