nyralabs / CrisperWhisper2.0_medium

CrisperWhisper 2.0

The most accurate verbatim speech recognition you can run in production: controllable, multilingual, and timed to the word.

Release post · Paper · Full documentation · Models · Benchmark · Benchmark repo

Most speech-to-text systems never actually decide whether to write down what was said or what was meant . They inherit that choice from their training data and apply it inconsistently. CrisperWhisper 2.0 makes it an explicit, controllable choice. One recording, two transcripts:

Verbatim , exactly what was said, in one consistent format: [um] so we we need to, to reschedule the th- thursday meeting to [uh] march third at nine thirty [laughter]

Intended , the clean version the speaker meant, with numbers, dates, and emails formatted the way you'd write them: So we need to reschedule the Thursday meeting to March 3 at 9:30.

On top of that:

• Word-level timings. Around 30 ms mean boundary error on read speech and 41 ms on conversational speech, the most precise word timing of any system we benchmarked, on both.
• Verbatimize. Upgrade transcripts you already have: given audio plus a trusted clean transcript, the model reproduces your content word-for-word and inserts only the disfluencies and vocal events actually present in the audio (rare-word recall jumps from 6.8% to 96.1% vs. re-transcribing). This turns the world's abundant clean corpora into verbatim ones, ready for TTS data, clinical speech analysis, and dataset construction.
• Multilingual. Verbatim and intended modes work across most languages Whisper supports. CrisperWhisper 2.0 tops the Nyra Verbatim Speech Benchmark leaderboard for disfluency F1 across ten languages, ahead of every closed-source alternative we tested.
• Seamless longform. Audio of any length, transcribed without the usual chunk-boundary artifacts: each window continues from the words already transcribed ( conditional continuation ), so there are no duplicated or dropped words at the seams and no fragile timestamp-token bookkeeping.
• Production inference. A CTranslate2 runtime with speculative decoding and built-in mitigation of Whisper's looping-hallucination failure mode.

Performance

The Nyra Verbatim Speech Benchmark scores fillers, repetitions, cut-offs, and vocal sounds as separate, typed metrics. Its headline number is disfluency F1 : how reliably a system writes down the disfluencies that were actually spoken, without inventing ones that weren't. Averaged over ten languages:

_{* CrisperWhisper 1.0 is English/German-only; its average covers those two languages. English and German use human-labeled evaluation sets; the other eight languages use synthetic verbatim sets. Per-language breakdowns and how the metric is computed are in the benchmark post .}

Word-timing accuracy

Mean absolute word-boundary error on read speech (TIMIT), lower is better:

_{Scored on exactly the words each system gets right. How the timings are extracted from supervised cross-attention, plus results on conversational speech, are in the aligner post .}

Install

# NVIDIA GPU (Linux): fastest, includes speculative decoding.
# An NVIDIA driver is all you need; CUDA libraries arrive via pip.
pip install "crisperwhisper[ct2]"

# Pure PyTorch: runs anywhere torch does (macOS, Windows, CPU)
pip install "crisperwhisper[transformers]"

Quickstart

from crisperwhisper import CrisperWhisperModel

model = CrisperWhisperModel()          # nyralabs/CrisperWhisper2.0_large
# or pick a size: CrisperWhisperModel("turbo")  # turbo / medium / small

# Verbatim transcription (default): every filler, repetition, stutter,
# false start, and vocal event
result = model.transcribe("meeting.wav", language="en")
print(result.text)

# Intended: the clean, readable version
clean = model.transcribe("meeting.wav", language="en", mode="intended")

# Word-level timestamps
result = model.transcribe("meeting.wav", language="en", word_timestamps=True)
for w in result.words:
    print(f"{w.start:6.2f}-{w.end:6.2f}  {w.word}")

# Verbatimize: upgrade an existing clean transcript with the
# disfluencies that are actually in the audio
result = model.verbatimize("clip.wav", "I think we should ship it Friday.")

Audio longer than 30 seconds is handled automatically (see longform below). The first load of a model downloads it from HuggingFace and, on the ct2 backend, converts it once into a local cache.

Models

The standard models are released under a non-commercial research license and are available for commercial licensing. The Pro models are available under commercial license only. For both, get in touch .

Faster inference: speculative decoding (ct2)

A small draft model proposes tokens and the main model verifies them. Same output, 1.3 to 1.4x faster:

model = CrisperWhisperModel("large", draft_model="turbo")
result = model.transcribe("meeting.wav", language="en",
                          speculative_decoding=True)

What else is in the box

Everything below works out of the box and is covered in depth in DOCS.md :

How it works

Each mechanism has a deep-dive post:

• Measuring verbatimness: the Nyra Verbatim Speech Benchmark . Typed metrics for fillers, repetitions, cut-offs, and vocal sounds instead of one opaque WER number.
• How we unlocked multilingual style-controlled transcription at scale . Verbatim/intended control across languages.
• Turning emergent cross-attention into a precise aligner . Supervising Whisper's alignment heads for word boundaries around 30 ms.
• Longform transcription with conditional continuation . Resolving window seams with the words already transcribed instead of fragile timestamp tokens.
• Closing the verbatim data gap with Verbatimize . Upgrading clean corpora into verbatim ones at scale.
• Faster inference and mitigating hallucinations . The CTranslate2 stack, speculative decoding, and the anti-looping decoder.

Documentation

DOCS.md covers the full API: backends and their trade-offs, every transcribe() option, dual-mode transcription, forced alignment, longform strategies, speculative-K tuning, hallucination-repair thresholds, quantization, model conversion, and the result object.

License

The model weights are released under the Nyra Health Non-Commercial Research License : free for research and other non-commercial use; any commercial use requires a commercial license. The Pro models are available under commercial license only. For commercial licensing of either, contact Nyra .