aufklarer / Smart-Turn-v3.2-CoreML

Smart Turn v3.2 — CoreML

End-of-turn detection for voice agents on Apple platforms. Given the last 8 seconds of the user's speech, the model returns the probability that the user has finished their turn, so an agent can reply promptly after a real endpoint and keep listening through a mid-sentence pause. It works on the raw audio (prosody, pace, intonation), not a transcript, and covers 23 languages.

The Whisper log-mel front-end is inside the model, including the zero-mean / unit-variance waveform normalisation the upstream model was trained with. Feed 16 kHz PCM and read one probability.

Model

Files

Performance

Accuracy on 1,000 clips from the upstream pipecat-ai/smart-turn-data-v3.2-test set (shard train-00000-of-00010.parquet , threshold 0.5). Latency is one 8-second window on Apple M5 Pro (macOS 26.5.2) (ONNX Runtime CPU with 2 intra-op threads, CoreML on CPU + Neural Engine). Higher accuracy / F1 is better; FPR is the share of unfinished turns wrongly cut off, FNR the share of finished turns the model kept waiting on.

Usage

import CoreML

let config = MLModelConfiguration()
config.computeUnits = .cpuAndNeuralEngine
let url = Bundle.main.url(forResource: "smart_turn", withExtension: "mlmodelc")!
let model = try MLModel(contentsOf: url, configuration: config)

// `turn` holds the user's current turn at 16 kHz; keep the last 128000 samples.
let window = try MLMultiArray(shape: [1, 128000], dataType: .float32)
let tail = turn.suffix(128000)
let offset = 128000 - tail.count
for (i, sample) in tail.enumerated() { window[offset + i] = NSNumber(value: sample) }
let output = try model.prediction(from: MLDictionaryFeatureProvider(dictionary: ["audio": window]))
let probability = output.featureValue(for: "probability")!.multiArrayValue![0].floatValue

# speech-swift CLI
speech turn --audio utterance.wav

Run it after Silero VAD reports a pause, on the whole current turn (up to 8 s). If the user resumes before the agent answers, run it again on the full turn.

Source

Converted from pipecat-ai/smart-turn-v3 (revision f766f81d3cfd , smart-turn-v3.2-gpu.onnx ), the open Smart Turn model from the Pipecat project, BSD-2-Clause. Training data and evaluation sets are published by Pipecat as pipecat-ai/smart-turn-data-v3.2-* .

Links

• speech-swift — Apple SDK
• Docs — install and CLI docs
• soniqo.audio — website
• blog — blog