Microsoft AI’s MAI-Transcribe-2 achieved the lowest transcription error rate in 18 of 21 languages in David AI’s new DAI-ASR-I18N public evaluation. The result matters because the benchmark uses 147 hours of unscripted conversation, a more demanding setting than clean, scripted audio and one that more closely resembles customer calls, interviews, meetings, and support recordings.
According to David AI’s DAI-ASR-I18N benchmark report, MAI-Transcribe-2 stayed below 10% transcription error in 16 languages. The evaluation also found that the model remained robust when separate speaker audio was mixed into a single channel. For teams that process multilingual spoken content, the results provide independent evidence that model performance can vary significantly by language and audio setup.
David AI compared eight commercial systems and six open-weight systems using normalized word error rate (WER) and character error rate (CER). WER measures the edits needed to turn a transcription into the reference text, while CER applies the same principle at character level, which can be useful for languages where word boundaries are handled differently. Lower error rates rank higher in the evaluation.
What David AI’s multilingual benchmark measures
DAI-ASR-I18N covers 21 languages and includes conversational speech as well as diarization, the task of separating or identifying speakers in an audio recording. David AI is releasing both the evaluation data and code publicly, including an MIT-licensed harness. That makes the benchmark more useful than a headline ranking alone: developers and buyers can inspect the methodology, reproduce tests, and compare systems against the same public framework.
The reported MAI-Transcribe-2 result is notable for its breadth. Rather than leading in just one or two major languages, it posted the best transcription result in most of the languages tested. However, it was not first everywhere. ElevenLabs Scribe v2 recorded the lowest error rate in the remaining three languages, including English.
| Benchmark result | MAI-Transcribe-2 | ElevenLabs Scribe v2 |
|---|---|---|
| Languages with the lowest reported transcription error | 18 of 21 | 3 of 21, including English |
| Error rate below 10% | 16 languages | Not specified in the supplied benchmark summary |
| Single-channel mixed-speaker robustness | Reported as robust | Not specified in the supplied benchmark summary |
The table should not be read as a universal product comparison. It reflects this particular public evaluation, its languages, recordings, normalization approach, and error metrics. A tool that performs best in one workflow may not be the best choice for every language, audio source, or downstream task.
Why conversational and mixed-channel audio matter
Many transcription tests can favor systems working from tidy recordings with clear speakers and minimal background noise. Real business audio is often less controlled. Calls may include interruptions, varying microphones, overlapping speech, and several people recorded on a shared channel.
The DAI-ASR-I18N result therefore has practical relevance beyond a model leaderboard. MAI-Transcribe-2’s reported resilience after speaker audio is combined into one channel suggests that its performance was not dependent on having each participant isolated in a separate audio track. That is useful context for organizations working with recordings that were not produced specifically for automated transcription.
At the same time, the benchmark summary does not establish performance for every possible recording condition. Teams should avoid assuming that a leading aggregate score guarantees the best output for their own calls, accents, vocabulary, recording equipment, or languages.
How to evaluate transcription tools for multilingual workflows
The benchmark gives buyers a credible starting point, but a controlled pilot remains essential. A practical evaluation should use representative recordings and measure the quality of the output needed for the actual workflow, not simply whether a transcript looks broadly readable.
Useful checks include:
- Test each language separately, especially languages that are material to customer service, sales, research, or internal operations.
- Use real audio samples that reflect typical call quality, speaker overlap, and microphone setups.
- Assess speaker handling when summaries, action items, or call notes need to be attributed to the right person.
- Measure correction effort, including the time staff spend fixing names, numbers, product terms, and important customer details.
- Review integration needs before rollout, such as whether transcripts must feed a CRM, help desk, archive, or analytics workflow.
The public dataset and MIT-licensed evaluation harness can help technically capable teams build more disciplined comparisons. For many businesses, the immediate takeaway is simpler: select a transcription model using evidence from the languages and recordings that matter most, rather than relying on an English-only test or a vendor demonstration.
The supplied benchmark results do not provide pricing, deployment terms, or a detailed availability comparison for MAI-Transcribe-2 and the other systems. Those factors still need to be evaluated separately alongside transcription accuracy, particularly when transcription is intended to become part of an operational workflow.
For businesses, better multilingual transcription can reduce the manual effort required to turn conversations into searchable records, follow-up tasks, and usable customer context. Scalevise can help map the right audio sources, select a practical model-testing approach, and connect reliable outputs to everyday systems through AI transcription workflow automation. Start by discussing an automation project with Scalevise.
Frequently Asked Questions
What is DAI-ASR-I18N?
DAI-ASR-I18N is David AI’s public multilingual benchmark dataset for conversational speech recognition and diarization. It covers 21 languages and 147 hours of unscripted conversations.
How did MAI-Transcribe-2 perform in the benchmark?
MAI-Transcribe-2 had the lowest reported transcription error rate in 18 of the 21 languages evaluated. It also remained below 10% error in 16 languages.
Which system led in English?
ElevenLabs Scribe v2 had the lowest reported error rate in the three languages not led by MAI-Transcribe-2, including English.
Does this benchmark prove MAI-Transcribe-2 is best for every transcription task?
No. The result applies to David AI’s evaluation dataset and methodology. Organizations should test representative recordings, languages, speaker conditions, and workflow requirements before choosing a system.
Conclusion
David AI’s DAI-ASR-I18N evaluation gives MAI-Transcribe-2 a strong independent multilingual result, with first-place transcription accuracy in 18 of 21 tested languages and reported robustness for mixed single-channel audio. The remaining results, including ElevenLabs Scribe v2’s lead in English and two other languages, reinforce the need to evaluate transcription tools against the specific languages and recording conditions a workflow requires.
