Spanish: the engine our own code made look broken

One of our earliest Spanish conclusions read: "Doubao collapses on hard Spanish, 98.7% error rate." On that clip it produced a garble of Chinese and English.

The problem turned out to be ours. We had been sending Doubao the language code es-ES (Spanish as spoken in Spain). Doubao doesn't accept that code. It didn't return an error either; it quietly fell back to Chinese. We switched to es-MX (Mexican Spanish) and reran all four clips. Doubao produced no invented sentences, kept most of the Argentine verb forms, and beat Speechmatics, the engine that had replaced it.

The original "collapse" was based on a single clip. Later, Doubao "collapses" in nine other languages turned out to have the same cause.

The test clips

Four film scenes from three countries:

• El Método (2005, Spain): a room of job candidates arguing over who the mole is, in Castilian Spanish.
• El crimen del Padre Amaro (2002, Mexico): priests arguing over a football match with the TV running underneath, until it turns into an accusation of colluding with outlaws.
• Nueve Reinas (2000, Argentina): a shouting match over an inheritance, plus a scene conning an old woman. Argentines use voseo, which changes how verbs are conjugated.
• La Comunidad (2000, Spain): neighbors crowding a doorway and yelling after finding a dead tenant's flat.

We treat La Comunidad as a hardest-tier sample; the other three fall between the middle and hardest tiers. The basic tier isn't covered (the tiers are explained in the first post).

The candidates

ElevenLabs, AssemblyAI, Doubao, Speechmatics (two models), Soniox, Gladia (two models), Deepgram, FunASR, Qwen, and later Gemini.

The six red lines, engine by engine

Flipped negation. The most common red line in Spanish. In El Método, Speechmatics turned "Quiero probar cosas nuevas" (I want to try new things) into "No quiero" (I don't want to), and "si hay un topo" (if there's a mole) into "Soy un topo" (I'm the mole). Gladia turned "Apaguen ya esa tele" (turn that TV off) into "¡No apaguen" (don't turn it off). Deepgram turned "no sé" (I don't know) into "lo sé" (I know).

Invented words and sentences. Gladia heard "nuestra situación laboral" (our work situation) as "nuestra situación, Laura," inventing a woman named Laura. Deepgram turned "porque está sola" (because she's alone) into "orquesta sola," conjuring up an orchestra.

Language drift. Speechmatics' Melia-1 model wrote "Soy abogado" (I'm a lawyer) as the Portuguese "Sou advogado," and switched the whole ending of La Comunidad into Portuguese. Spanish and Portuguese are close enough that setting the language explicitly didn't stop it.

Dropped passages. Deepgram skipped the overlapping football chatter at the start of Padre Amaro and cut off the climax of the shouting at the end of La Comunidad.

Flattened accents. Not a red line, but it matters for Latin American interviews. Soniox changed the Argentine "Oíme / hablás / andás" into the neutral "Óyeme / hablas / andas." ElevenLabs and AssemblyAI kept the originals.

Repetition. Gladia showed early signs of repeating the same word on Spanish clips. None of the others had that on these four clips.

There was reassuring news too. La Comunidad is the loudest scene, yet six engines converged on the same dialogue and none invented anything. The subtitle has a single line, "Habrán dejado el grifo abierto," and all six transcribed the full sentence as actually spoken, which is more than the subtitle has.

What we chose

Main track: ElevenLabs Scribe v2. Solid on all three accents, the best at keeping Argentine voseo, and no invented sentences or flipped meanings on any of the four clips.

Reference tracks, in priority order:

1. AssemblyAI (universal-2): no invented sentences and no flips on any clip; the cleanest and most complete of the group.
2. Doubao (with the es-MX code): no invented sentences and no collapses across four clips, and most Argentine verb forms intact. It recovered real words that the compressed subtitles had dropped (complacencias, rumbos), and it got right the line Deepgram turned into an orchestra.
3. Gemini: added on August 26, 2026.

Speechmatics is now an emergency fallback. It doesn't invent, but it flipped two lines in El Método and was the weakest on the Argentine clip.

Evidence against our choice

• Doubao flipped a line too, at the same spot as Speechmatics. In El Método, Doubao also wrote "Quiero" as "No quiero." Speechmatics isn't running in production, so this particular line won't get two wrong votes. But we've seen several engines go wrong at the same spot in five languages, and when that happens, a simple majority vote picks the wrong answer. That's why our merge step doesn't settle negation by majority: without hard evidence, the line is flagged for a person.
• Gemini has flips of its own in Spanish. In Padre Amaro it turned "turn off the TV" into "don't turn off the TV" and "I don't know" into "I know." The other three tracks got both right. In La Comunidad it changed "left the gas on" to "left the house on," which no longer fits the next line about everyone getting blown sky-high.
• Qwen was clean in Spanish, and we still passed on it. Across four clips it had no loops and no invented sentences, and it kept every Argentine verb form. We left it out because it has no word-level timestamps, belongs to the same large-model family as Gemini, and was less steady than our current three on clean audio.

Open questions and limits

• In Latin America we only tested Mexico and Argentina. There are no clips for Caribbean, Colombian, Chilean or other accents.
• All four clips are from films. None is a real interview, and none is in the basic tier.
• When Doubao gets a language code it doesn't accept, it doesn't raise an error; it just outputs garble. The code is correct now, but this kind of problem never shows up in error logs. You only catch it by reading the output.