English: two reference tracks that fail in different places

In a bar interrogation scene from The Departed, someone says "I don't know what they do." Doubao transcribed it as "You know what they do," which flips the meaning. AssemblyAI got that line right. But in the same scene, AssemblyAI dropped an entire key denial, "I am not a fucking cop!", which the other two tracks can fill in.

After reading all four clips line by line, we found these two engines almost never trip over the same spot. That's the principle behind our English reference tracks.

The test clips

Four scenes from American and British films, each built around one kind of difficulty:

• Gosford Park (2001): upper-class British accents, a servants' dinner below stairs, quick exchanges and interruptions over background noise.
• The Social Network (2010): clean, fast American speech in a deposition, dense with legal terms and names.
• The Departed (2006): Boston accents, bar noise, slang and swearing, low menacing questioning.
• August: Osage County (2013): a whole family talking over each other at the dinner table.

On the three tiers from the first post in this series, The Social Network sits near the middle tier and the other three are in the hardest. The basic tier isn't covered.

The candidates

ElevenLabs, AssemblyAI, Doubao, Speechmatics (two models), Soniox, Deepgram, Gladia (two models) and Gemini, plus two Qwen models and FunASR. iFlytek was tried on the first clip only; it has no English-only mode, so we stopped there.

The six red lines, engine by engine

Invented sentences. In Gosford Park, "I heard Lady Sylvia spoke out of turn" came out of Soniox as "My hand, ladies, don't be a stroke out at 20 minutes PC." Deepgram added two names nobody said in the same clip, "Sister Fredrick" and "Percy." One of the Qwen models wrote a full sentence into The Departed that isn't in the audio: "buy some bananas."

Flipped negation. This was the most common red line in English. In The Departed, Soniox and Deepgram both dropped the "not" from "you're still not a cop." Speechmatics turned "Unless I stop them" into "as I stop them." Doubao's flip is the one at the top of this post. We gave Qwen a stricter prompt that explicitly told it to keep negations. It dropped the "not" anyway.

Loops and collapse. In August: Osage County, someone shouts "Eat your fish" five times. Gladia transcribed it once, then cut off, losing about nine words.

Dropped passages. Deepgram skipped a large chunk of Gosford Park and made more errors there than any other engine.

Language drift. None on these four clips.

Inserted content. Besides the invented names above, Speechmatics turned "a 3L" (a third-year law student) into "a three year old."

What we chose

Main track: ElevenLabs Scribe v2. It was the cleanest or second-cleanest on three of the four clips and didn't fall apart on any of them.

Reference tracks, in priority order:

1. AssemblyAI (universal-2): no invented sentences and no flips across all four clips, with the most solid overall structure. Its one miss was the denial in The Departed, and that was an omission, not an invention.
2. Doubao: no invented sentences on any clip. It caught all five "Eat your fish" shouts, and on that clip it was more accurate than AssemblyAI. It's built differently from AssemblyAI and makes its mistakes in different places.
3. Gemini: added on August 26, 2026. When we tested it across 26 languages, English landed in the group where it could go straight in. On the August: Osage County shouting match it had no red lines.

Evidence against our choice

• The main track made Doubao's exact mistake. When we reran the clips later, ElevenLabs also produced "You know what they do." In the first run it had garbled a whole line in Gosford Park; on the rerun, that error was gone. The same input doesn't give the same output every time, so we judge engines by the kinds of errors they make, not by how many we happened to count.
• Gemini flips negations too. When we first tested it as a backup main track, it turned "Yes. Yes, yes." in The Departed into "No. Yes! Yes, yes!" and "I don't know" into "I know." In the August round, it rewrote the Lady Sylvia line in Gosford Park into two unrelated sentences. We added it because, as a large language model, it fails differently from acoustic engines. Yet on that one line in The Departed, Doubao, Gemini and the rerun of ElevenLabs all got it wrong. Engine errors aren't always independent, and we don't have a good fix for that yet.
• The lowest-WER engines didn't make the cut. Speechmatics had the lowest WER on Gosford Park (11.4%), and Soniox had the lowest on August: Osage County (4.7%). Both invented sentences or flipped negations on other clips, so we passed on them.

Open questions and limits

• All four clips are from films. None is a real interview, and none is in the basic tier.
• Accents covered: British and American, including Boston. No other English accents were tested.
• We've seen Doubao's flip only once, but it's the most dangerous kind of error. After the rerun, AssemblyAI was the only one of the four tracks that got that line right. That's why, on negation and reversed meaning, our merge step doesn't go with the majority: without hard evidence from context, the line is flagged for a person to check.