Scottish accent vs 8 local speech-to-text models: scores, mistakes and timings

The short answer: Voxtral scored highest

Mistral’s Voxtral Mini 4B Realtime had the highest overall transcript score in this test: 9.87 out of 10. NVIDIA’s Nemotron 3.5 Transcribe was second at 8.97 and Canary-Qwen third at 8.93. Qwen3-ASR and Whisper large-v3 tied for fourth at 8.88, Parakeet was sixth at 8.79, the small Nemotron streaming model seventh at 8.62 and Whisper large-v3 Turbo eighth at 8.57.

Read that order with two caveats. The score is provisional: the reference transcript every model was marked against started from Voxtral’s own output before it was reviewed, which can favour Voxtral. And Voxtral was also the slowest, taking about five minutes on its first pass over a two-minute file. Apart from Voxtral, the scores are close, so the individual mistakes below are the better guide to which model suits you.

What we tested

These are downloadable speech recognition models: they take audio and produce a transcript. Only that step was tested, with no chatbot afterwards rewriting the answer and making a transcript look better than it was. The comparison uses the raw saved transcripts.

Every model got the same two-minute recording of Luke: his normal voice, an attempt at an English accent, more natural Scottish speech, place names, “purple burglar”, numbers and a stretch of unscripted talking. He stumbles in the English-accent passage, so the recording isn’t a controlled test of which accent a model prefers.

Everything ran on the same M5 Pro MacBook with 48 GB of RAM, using local Mac implementations of each model, some of them compressed versions (Nemotron Transcribe, for example, ran as a 4-bit build). These are the results those particular setups gave on this recording, not a claim that every app using these models will produce the same output. Accents vary, and so can the output from one run to the next.

Whisper large-v3 and Turbo (OpenAI)

Whisper large-v3 and large-v3 Turbo are two versions of the same OpenAI model family, so the pair also compares the full model with a version intended to be faster. On the normal passage both outputs are recognisable.

The test sentence was “While driving from Ecclefechan to Glesga, then meeting somebody in Milngavie.” Large-v3 wrote Ekelveken and Mogai, and Turbo wrote Ekofekin and Mo’Gai: close to the sound, but not the places. Glasgow came through as Glesga, which is how Luke said it, so that counts as correct.

The clearest difference was purple burglar. The full model wrote purple burglar; Turbo wrote purple burger, then purple burger alarm. Luke stumbles there, so the audio matters too, but it shows why a transcript can look perfectly reasonable until you compare it with what was actually said.

NVIDIA Parakeet TDT 0.6B v3

NVIDIA describes Parakeet TDT 0.6B v3 as a high-throughput transcription model, and it supports 25 European languages. We used a local MLX version on the Mac, and it got through the whole recording in about five seconds on its first pass, excluding loading.

But the place-name passage disappeared from its output: it goes from the Scottish speech straight to purple burglar without transcribing the route between them. If you were dictating an address, you could fix a misspelling, but you would first have to notice that the whole line was missing. Parakeet’s speed is worth knowing about; check its output closely all the same.

Qwen3-ASR (Qwen, Alibaba)

Qwen3-ASR comes from the Qwen team at Alibaba. One reason to include it: a British accent study (“Mind the Accent Gap”, on arXiv) tested Qwen3-ASR in offline mode on spoken Scottish financial queries, and it had the lowest Scottish word error rate among the configurations in that comparison. That is a different task, setup and metric, so it makes Qwen a strong candidate rather than predicting this test.

Here the place names still came back wrong, and purple burglar became purple burger, followed by purple burger alarm. It finished its first pass in just over five seconds, excluding loading.

NVIDIA Canary-Qwen 2.5B

Despite the name, Canary-Qwen 2.5B is a separate NVIDIA speech model, not the Qwen3-ASR model above. Our Mac adapter processed the recording in 25-second chunks, and the purple burglar section crosses a chunk boundary: the saved output has Burpo Burger, followed in the next chunk by burglar alert.

That isn’t evidence that the model is bad with Scottish accents. It shows that this particular setup needs more work before you rely on it for dictation.

NVIDIA Nemotron 3.5 ASR streaming

The small Nemotron 3.5 ASR streaming model is built to process speech incrementally. We gave it the recorded file, so this round doesn’t measure how quickly words would appear while someone is talking, and there is no saved time for it.

It produced a readable transcript of the general conversation, but the numbers line is a useful example. Luke said: “I said 13, not 30. It costs £3.50 and I need 16 of them. Actually, make it 60.” Where the reviewed reference has “I need 16 of them”, Nemotron streaming wrote 60, then kept the correction to 60 afterwards. In a message with a quantity in it, that is the sort of error you would want to catch before sending.

NVIDIA Nemotron 3.5 Transcribe

Nemotron 3.5 Transcribe is a much larger NVIDIA model. NVIDIA reports a 4.71% average English word error rate across nine public test sets; that is its published benchmark, not a Scottish-accent score. We ran a 4-bit early-access Mac version, so the precision and implementation differ too. It processed the file in about 17 seconds after loading.

It got purple burglar right. But Ecclefechan became Eko Faking, Milngavie became Mogai, and it also wrote 60 in the quantity line. The larger model still leaves specific words to correct.

Mistral Voxtral Mini 4B Realtime

Voxtral Mini 4B Realtime is designed for real-time transcription, but here it was fed the saved recording too. Our full-precision Mac pass took about five minutes to finish the two-minute file. These are first runs, not a matched warm speed benchmark, but that is a noticeable wait.

Voxtral wrote “13, not 30”, where most of the other outputs have “30, not 30”, and it kept 16 before the correction to 60. It still missed Ecclefechan and Milngavie, so its row isn’t clean either.

Provisional scores and timing

Each model’s overall transcript score starts at 10 and loses points for words that were changed, missed or added, measured against the same reviewed reference for all eight. It covers the whole recording, including the stumbles and the talk between the challenges. Because that reference was derived from Voxtral’s output, treat the order as provisional rather than an independent accuracy ranking.

The times are first passes over the file, excluding loading, with different implementations and precision and no matched warm-up, so they aren’t a controlled speed ranking or a live-latency test. Speed isn’t part of the score: there is no saved time for the small Nemotron model, and adding speed points would have made that missing measurement part of the rank.

Provisional transcript scores out of 10, with the first-pass times reported in the video:

1. Voxtral Mini 4B Realtime (Mistral): 9.87, about 5 minutes
2. Nemotron 3.5 Transcribe (NVIDIA): 8.97, about 17 seconds after loading
3. Canary-Qwen 2.5B (NVIDIA): 8.93
4. Qwen3-ASR (Qwen): 8.88, just over 5 seconds
5. Whisper large-v3 (OpenAI): 8.88, tied with Qwen3-ASR
6. Parakeet TDT 0.6B v3 (NVIDIA): 8.79, about 5 seconds
7. Nemotron 3.5 ASR streaming (NVIDIA): 8.62, no saved time
8. Whisper large-v3 Turbo (OpenAI): 8.57

Look at the mistakes before you choose

The scores are close enough that the actual errors matter more. Qwen3-ASR and Whisper large-v3 get the same score but don’t make the same mistakes, and Voxtral’s top score doesn’t remove its five-minute wait. The order is a provisional way to compare general accuracy; which model suits you depends on your case.

With the place names Luke confirmed, none of the eight outputs spelled Ecclefechan or Milngavie correctly. Seven recovered Glasgow (as Glasgow or Glesga), and Parakeet left out the place passage altogether. That is a clear result from this recording, and it is exactly the kind of vocabulary someone uses when speaking normally.

Test your own voice

One recording of one voice can’t tell you how a model will handle yours. To run your own comparison:

1. Record one file in your normal voice that includes ordinary speech plus the names, places and quantities you actually use.
2. Write down exactly what you said, and check it against the audio, so every model is marked against the same reference.
3. Run every model on that same file and keep each raw transcript, before any chatbot or app cleans it up.
4. Check for missing passages as well as misspellings: a dropped line is easier to overlook than a wrong word.
5. Record each model’s version, precision and implementation, and time loading separately from the transcription itself.
6. Use this guide’s scores as a starting point for what to try, not as an independent benchmark.

Where to find the models

Each model’s own page is linked on this page: OpenAI’s Whisper repository on GitHub (large-v3 and Turbo), NVIDIA’s Parakeet TDT 0.6B v3, Canary-Qwen 2.5B and Nemotron 3.5 ASR streaming model cards and its Nemotron 3.5 Transcribe research page, the Qwen3-ASR model card, and Mistral’s Voxtral Mini 4B Realtime model card. The British accent study that put Qwen3-ASR on the list is linked too.

The video ran local Mac (MLX) versions of these models, some of them compressed, rather than the original releases, so check which version and precision you are installing before comparing your results with these.

The takeaway

On this one Scottish recording, Voxtral had the highest provisional transcript score, 9.87 out of 10, but took about five minutes, and the reference it was scored against started from its own output. No model spelled Ecclefechan or Milngavie, Parakeet dropped the place passage, and several changed a quantity. Look at the mistakes on your own voice before you pick one.