Part of: 27 languages, one by one
Hungarian: why we kept the engine that wrote "ticket" 23 times
· Zeyu Si
On this page
The Hungarian film Kontroll (2003) has a brawl over tickets on the metro. The inspector keeps shouting "Jegy!" (ticket). AssemblyAI wrote the word 23 times in a row, and the whole stretch of dialogue after it disappeared.
In a different clip, fast back-and-forth ordering at a snack bar, the same engine didn't loop once. We kept it for Hungarian. The case for keeping it turned out weaker than we first thought, though.
The test clips
Five clips, all from films:
| Clip | Scene | Difficulty |
|---|---|---|
| Kontroll (2003) | Ticket-inspection brawl on the metro, several people shouting at once, swearing, background noise | Extreme |
| Valami Amerika (2002) | A party, music louder than the voices, fast exchanges | Extreme |
| Üvegtigris (2001), part 1 | A tyre shop in the countryside, five-plus people yelling, an engine running | Extreme |
| Üvegtigris, part 2 | Two people talking plus a monologue, little background noise | Basic |
| Üvegtigris, part 3 | Ordering at a snack bar: bickering, a walkie-talkie, someone cutting in. Speakers change fast, but no long stretches of overlap | Medium-high |
The last three come from the same film. Complete, clean sources for recent Hungarian films are hard to find; the only full-length copy of one film we wanted was stuck behind an age gate.
Engines we tested
ElevenLabs Scribe v2, AssemblyAI, Speechmatics, Soniox, Deepgram, Gladia, FunASR and Doubao, with Gemini added later.
Hungarian isn't one of Doubao's supported languages, and all five clips came back as garbage. Deepgram and FunASR produced far too little text, roughly a tenth and three-tenths of the reference word count, which means most of the audio simply wasn't transcribed.
The red lines
Loops: the "Jegy" run is one. In the tyre-shop clip, AssemblyAI repeated "Mi az hogy mi között van" 18 times, swallowing six different lines. Gemini also had several runs of repetition in Kontroll.
Negation flips: Speechmatics turned "Ne csináljátok a fotót" (don't take that photo) into "Csináljátok a fotót" (take it). AssemblyAI turned "this party is amazing" into "why isn't this party any good". At the snack bar it heard megvennék (I'd buy it) as megvernék (I'd beat him up). Gemini turned "don't butt in" into "go ahead, butt in".
Hallucinated sentences: where the party music drowned out the voices, Speechmatics produced a burst of laughter and a sentence that aren't in the film. AssemblyAI replaced a whole line in Kontroll with a different one. Gemini added a line in the tyre shop, "we'll pay for the ride", that nobody said.
Dropped passages: at the snack bar, AssemblyAI missed two walkie-talkie exchanges.
Inserted content: in the same clip, Gemini pushed the walkie-talkie call into the middle of the conversation six times.
Language drift: Doubao only.
ElevenLabs had no loops and no flips across all five clips. Its mistakes were sound-alike substitutions, such as színed for szénád.
What we chose
- Primary engine: ElevenLabs Scribe v2.
- Reference engines: AssemblyAI, Soniox, Gemini.
AssemblyAI stayed because its loops only happened in long, evenly matched shouting matches. In the two-person dialogue and the snack-bar clip it didn't loop or drop whole stretches. Real interviews rarely involve several people yelling over each other for long stretches.
Soniox replaced Speechmatics, which invents sentences in noise and has dropped a negation; Speechmatics is now an emergency fallback. On review, Soniox beat AssemblyAI on all five clips, and when it failed, it tended to produce visibly broken fragments.
Gemini was added to every language in August. It fails differently from the acoustic engines, so the two kinds can check each other.
Evidence against our choices
Hungarian has a fair amount of it.
- The claim that AssemblyAI is clean on conversation took a hit from its own error: "buy" became "beat up" in the snack-bar clip, the one closest to an interview.
- The review found Soniox better than AssemblyAI on all five clips, yet our configuration still lists AssemblyAI ahead of Soniox. The order runs against the evidence.
- Gemini had only one conversational clip to judge, and it was clean there. In the two group scenes it had a flip, an invented sentence and several loops, and at the snack bar it inserted the walkie-talkie call six times.
- On WER alone, Speechmatics scored lowest on both the two-person clip and the snack bar (20.5% and 33.5%, against 27.6% and 36.3% for ElevenLabs). We didn't choose on that number; see WER lies.
What's still open
- One conversational clip isn't enough evidence for Gemini in Hungarian. It needs more testing.
- Three of the five clips come from one film, which limits the range of scenes.
- Whether to reorder the reference engines to match the review hasn't been decided.
Our Hungarian transcription page
Written by Zeyu Si