2026 ASR benchmark: text, timing and speaker trade-offs
Short answer: no model won every category on this sample. The strongest observed text result did not include speaker labels, while integrated speaker-capable systems made different segmentation or transcription errors. Native word timing also varied by engine, so timing and diarization must be evaluated separately from text quality.
Important: this was a screening experiment on one 15.093-second Chinese animated-dialogue clip. It is not a universal accuracy ranking, and this page does not publish a formal word error rate, character error rate or diarization error rate.
Question and scope
The experiment asked a practical product question: when the same short recording is sent to different speech-recognition systems, how do the returned text, processing time, word or character timing and automatic speaker output differ?
The clip contains rapid character and voice changes. Visual review suggested four characters and at least three plausible speaking voices. There was no independently human-labelled speaker timeline, so speaker behavior was assessed qualitatively by turn consistency rather than by a formal diarization score.
Method
- Use one unchanged source. Every result in the primary table used the same 15.093-second source clip.
- Record wall time. Cloud figures include upload, network, scheduling and polling. Local figures identify warm inference where the model was already loaded.
- Review text against a consensus transcript. The consensus was assembled through repeated listening, but it was not a separately commissioned professional ground truth.
- Inspect timing and speaker output independently. A model could return strong text but no speaker labels, or return detailed timing with weaker text.
- Keep caveats with each result. A selected observation applies to this clip and configuration, not to every recording in the model's supported languages.
Selected same-sample observations
| Model or pipeline | Observed processing time | Text observation on this clip | Timing returned | Speaker output |
|---|---|---|---|---|
| Qwen3-ASR-Flash-Filetrans | 4.751 s | No material word error found in the qualitative review; punctuation was clean. | Native word timing | No |
| Fun-ASR-MTL | 5.739 s | Close to the consensus, with some turn and punctuation merging. | Native word timing | Automatic labels, but not fully reliable on this short clip |
| Paraformer-v2 | 4.827 s | Core words were mostly correct, with a visibly incorrect sentence split. | Word timing | Automatic labels returned |
| iFlytek LFASR | About 5 s | Near-consensus text in this run. | Word and sentence timing | Automatic mode returned only one speaker |
| Speechmatics Standard | 6.351 s | Multiple material Chinese transcription errors were observed. | Word timing | Automatic mode returned one speaker |
| Whisper large-v3 INT8, local CPU | 8.184 s warm | One clear lexical difference was observed in the reviewed transcript. | Word timing | No integrated speaker labels |
| Qwen3-ASR 0.6B INT8 ONNX, local CPU | 1.602 s warm | Very clean text on this short sample. | No native workspace timing in this run | No |
| SenseVoiceSmall INT8 ONNX, local CPU | 0.200 s warm | One material lexical difference was observed. | Character timing points | No integrated speaker labels |
Times are observations from the recorded environment, not vendor service-level commitments. Network, queue load, audio length and hardware can change end-to-end latency.
What the experiment shows
Text quality and workspace completeness are different questions
The cleanest text result on this sample did not include integrated speaker labels. Systems that returned text, timing and speakers in one response were more feature-complete, but some merged turns or returned an implausible speaker count. Selecting a production route therefore requires separate acceptance thresholds for text, timing and speakers.
Word timing is not a universal capability
Some engines returned native word or character timing; another strong local text result did not. A transcript product should not manufacture precise word timing when the engine only returns segment timing. Media2Text uses word-level timing when reliable data is returned and otherwise keeps playback and subtitle generation at segment level.
Automatic speaker count is an estimate
A separate local diarization screening on the same clip produced two detected speakers at one clustering threshold and three at another. That sensitivity is why Media2Text exposes generic, editable speaker labels instead of presenting automatic labels as verified identities.
Fast local inference is not the same as a complete service
The fastest local runs were model-ready inference measurements. They did not include all of the work needed for a production service, such as file transfer, queuing, timing alignment, speaker processing, retries and durable result storage.
Limitations
- The primary comparison used one short Chinese animated-dialogue clip; it does not establish multilingual or long-form performance.
- The consensus transcript was carefully reviewed but was not an independently labelled benchmark corpus.
- There was no human-labelled speaker timeline, so no formal diarization error rate is reported.
- Cloud and local timing measurements include different components and should not be treated as a hardware-neutral speed ranking.
- Model versions, service configurations and provider behavior can change after the experiment date.
- The source media is not redistributed on this page because Media2Text does not claim redistribution rights for it.
What a stronger follow-up requires
A production-grade benchmark should use rights-cleared, human-labelled recordings across interviews, meetings, overlapping speech, noisy mobile audio, multiple accents and every launch language. It should report CER or WER, diarization error rate, speaker-count error, timestamp drift, wall time, queue time and failure rate. Results should be versioned with model identifiers and the test date.
Primary model references
- Qwen3-ASR official repository
- Fun-ASR official repository
- OpenAI Whisper official repository
- FunASR toolkit repository
For the product behavior built around these findings, see Audio to Text, Video to Text and the security and data-flow page.