desert-ant-labs / align

Align

Timestamps that land on the word.

Word-timestamp refinement for Apple's SpeechAnalyzer pipeline.

• SDKs, install and examples: https://github.com/Desert-Ant-Labs/desert-ant-core/blob/main/docs/models/align.md

Corrects the word-level timings that Apple's SpeechTranscriber and SpeechAnalyzer return, without replacing them. Align observes the same audio the analyzer already receives, runs a small Core ML cascade on the CPU and Neural Engine, and returns the familiar result surface with tightened audioTimeRange values. The models are tiny ( 700KB compiled Core ML) and refine a typical result in a few milliseconds on device.

Apple: "world" 2.61-3.04s ➜ Align: "world" 2.57-2.98s

Try it

Install

Swift ( requirements )

.package(url: "https://github.com/Desert-Ant-Labs/desert-ant-core.git", from: "3.1.0")

Then add the Align product to your target.

Files

The compiled .mlmodelc stages, mel_filters.bin , calibrator.bin , and refiner_config.json are exactly what the Swift SDK bundles. The .pt checkpoints are the training-run weights.

Architecture

A two-stage coarse-to-fine cascade over a log-mel spectrogram, refining one boundary at a time:

• Frontend : an Accelerate/vDSP log-mel spectrogram of the same audio Apple transcribes.
• Coarse stage : a compact convolutional model searches a 2.4 s context around Apple's proposed boundary and predicts a distribution over frames.
• Fine stage : a second model re-searches a 0.8 s crop recentered on the coarse prediction for a tighter estimate.
• Lexical conditioning : UTF-8 byte features of the neighboring words plus a language id let a single model cover all nine languages.
• Calibrator : a small gradient-boosted-tree policy maps coarse/fine uncertainty features to a final correction, fit only on the validation split to reduce large regressions.
• Structural fallback : boundaries whose correction would be invalid, hit the search-window edge, or lack streaming context keep Apple's original timestamp.

Each stage runs fixed batch-16 on CPU + Neural Engine. Total parameters are about 117k per stage.

Inputs and outputs

• Input: mono audio plus Apple's recognized words with their proposed start/end times.
• Output: the same words with corrected start/end times, or Apple's original time when a correction is not structurally safe.

Accuracy

Evaluated on the exact Swift runtime and these bundled Core ML models over 223 clean and 210 noisy group-held-out recordings across all nine languages, against forced-alignment references.

Error is mean absolute distance from the reference boundary. Align roughly halves Apple's typical error and removes most of its large mistakes.

Languages

English, Spanish, French, Italian, Portuguese, German, Japanese, Korean, and Chinese. A locale outside this set is passed through unchanged.

Limitations

• References are machine forced-alignment estimates, not human annotations, so the figures show a large, consistent reduction of Apple's timing error rather than sample-accurate ground truth.
• A learned correction is not guaranteed to improve every boundary; the structural fallback keeps Apple's timestamp when a correction looks unsafe but cannot catch every plausible-looking error.
• English, Italian, Japanese, and Korean are the weakest languages under the current reference convention.

Built on

• FLEURS (CC BY 4.0): multilingual training audio.
• Qwen3-ForcedAligner-0.6B (Apache-2.0): primary word-boundary references for all nine languages.
• OWSM-CTC v4 1B (CC BY 4.0): gross alignment-outlier check where validation agreement is stable.
• Genuine Apple SpeechAnalyzer proposals collected on macOS 26.

See THIRD_PARTY_NOTICES.md . None of these systems are redistributed here.

License

Desert Ant Labs Source-Available License . Free for most apps, and a commercial license is required at scale. Full terms are at the link. Licensing: [email protected] .

See THIRD_PARTY_NOTICES.md .

Citation

@software{align_2026,
  title  = {Align: Word-timestamp refinement for Apple's SpeechAnalyzer pipeline},
  author = {Desert Ant Labs},
  year   = {2026},
  url    = {https://huggingface.co/desert-ant-labs/align},
}

---

© 2026 Desert Ant Labs · https://desertant.com