Ukrainian: 54 seconds of silence, turned into a lecture

A Ukrainian film has a living-room scene where a TV is on in the background and nobody speaks for about a minute. The subtitles mark it with a single line: [TV playing in the background].

Soniox wrote 54 seconds of content into that gap, something about poisonous plants, stuck in a small loop partway through. Speechmatics filled 29 seconds. Maybe they really were picking up the TV. For an interview transcript it makes no difference: a long passage appears out of nowhere and reads as if the interviewee said it.

It was the biggest fabricated insertion in our entire test set. Soniox has replaced other engines in several languages. It isn't used for Ukrainian, and those 54 seconds are the reason.

The test clips: finding them was half the work

• My Thoughts Are Silent (Мої думки тихі, 2019), three scenes: a smuggling interrogation at the border (heavy noise, strong Transcarpathian dialect; hardest tier); the lipstick search that follows (faster, more overlap; hardest tier); a teacher's living room (standard Ukrainian, clear; baseline tier).
• Stop-Zemlia (Стоп-Земля, 2021): interviews in a classroom plus street scenes. Fast speech, frequent speaker changes, street noise. Mid-to-high tier.

Ukrainian material is hard to assemble. Recent films are tightly licensed, war films often have no full release or no subtitles, and films set in the Donbas are half in Russian. The first three clips had to come from one film, so we took the fourth from a different one to make sure the findings weren't about a single movie.

Engines we tested

ElevenLabs Scribe v2, AssemblyAI, Speechmatics and Doubao (uk-UA). Transcripts from Soniox, Deepgram and Qwen came from a separate all-engine run and were compared during the review. Gemini was added in August 2026.

Red lines, engine by engine

ElevenLabs: no loops, no hallucinations, no flips across all four clips. It handled the dialect best, e.g. файні трави ("good herbs") and тя роздеру як жабу ("I'll tear you apart like a frog").

AssemblyAI: at the end of the lipstick scene it repeated one line, "Треба цього конючку", three times. In the same scene it turned подозрітєльний ("suspicious") into правдивий ("truthful"), close to a reversal, and dropped a whole passage where rules are read out. In the fourth clip, "А якщо не вступиш?" ("what if you don't get in?") became наступиш ("step on").

Doubao: the original report said it had no reversals. The review found three reversal-level errors: "suspicious" became приємний ("pleasant"), "don't get in" became "step on", and "he didn't go there" lost its negation. On the street it also inserted a swear word and the word російський ("Russian") that nobody said.

Speechmatics: turned the TV noise into dialogue; dropped negations twice, including "Це скоріше не пушка" ("this is more like not a gun") becoming "this is more like a gun"; and wrote сенсорну ("sensory") for стресорну ("stress") reaction.

Soniox: see the 54 seconds above.

What we chose, and why

Primary: ElevenLabs Scribe v2. The steadiest across all four clips, with none of the three failure types.

Reference 1: AssemblyAI. Its loop happened once, only where heavy dialect, fast speech and dense overlap all came together. In the clear living-room scene and the interview-style fourth clip it was clean.

Reference 2: Doubao. Coherent Ukrainian on all four clips, no loops, no garbage output.

Reference 3: Gemini. This one went back and forth; see the next section.

Evidence against our choice

• Our two reference tracks can be wrong together. On "suspicious" and "don't get in", AssemblyAI and Doubao failed at the same point in the same direction, and Soniox, the engine we rejected, got both right. When the two reference tracks agree, that doesn't make them right, especially on negations and on whether something is good or bad.
• The original report was too kind to the incumbents. Doubao's reversals went from zero to three. The claim that "only ElevenLabs got 'don't get in' right" didn't hold: ElevenLabs, Speechmatics, Soniox, Deepgram and Qwen all did.
• Gemini did poorly on the film clips. In the lipstick scene it switched to Russian midway (базаришь что такое) and inserted a swear word. In the clear living-room scene it added 40 words that none of the other three produced. At one point we listed Ukrainian as "don't add". Then we ran a five-minute real political interview podcast. No hallucinations, no flips, no loops, and one word it got right where ElevenLabs didn't (двієчник, "poor student"). It also left out two direct quotes, about 30 words in all, and turned Сповідь ("confession") into Ковід ("Covid"). That podcast had no human reference transcript; we only compared Gemini against ElevenLabs.

Still open

• Three of the four clips come from one film. The source base is narrow.
• The first three clips were initially judged only on the shape of the output; line-by-line reading came in a later review.
• No real research interviews yet. Gemini's inclusion rests on a single five-minute podcast.