Portuguese: three flipped lines, and a key sentence rescued

In an in-depth interview from Portuguese television, the most important sentence contains "no chão da cozinha": on the kitchen floor. Our main track, ElevenLabs, wrote "no som da coxinha," a string of sounds with no meaning. Gemini got it right.

In the same interview, Gemini turned "nunca disse isto" (I never said that) into "Tu disseste isto" (you said that). In each of the three conversations we added to test it, Gemini flipped one line like this.

Whether to include Gemini was the most argued-over decision in Portuguese.

The test clips

Three film scenes for choosing the acoustic engines, all Brazilian Portuguese, two to three minutes each:

• Meu Nome Não É Johnny (2008, Rio): a party with a drug deal and people heckling, dense slang, overlapping voices.
• Que Horas Ela Volta? (2015, São Paulo): six people by a pool, flirting that suddenly turns into everyone shouting orders.
• O Auto da Compadecida (2000, the Northeast): a gang of bandits discovers who someone really is, five or six people talking fast in a strong northeastern accent.

Three real conversations added to test Gemini, five minutes each, with no reference transcript; we compared Gemini with ElevenLabs line by line. One is a Brazilian political interview, one an in-depth interview on Portuguese television (with music beds between segments), and one a podcast with three people talking over each other. The last two are in European Portuguese.

On the three tiers from the first post, all three film scenes are in the hardest tier. The conversations are closer to real interviews, but without a reference transcript we can only check the engines against each other.

The candidates

ElevenLabs, AssemblyAI, Doubao, Speechmatics (two models), Soniox, Gladia, Deepgram, FunASR, Qwen, and Gemini.

The six red lines, engine by engine

Invented sentences. At a transition in the São Paulo crowd scene, AssemblyAI produced an English line, "That's so good." In the Northeast clip it turned "Oxente, e agora é de dois" (hey, now it's two) into "Oxe o Diogo agora é de 12," adding a name and getting the number wrong. Speechmatics inserted "Não mais na escuridão" (no more in the darkness) into the Rio clip and collapsed a line in the São Paulo clip into "Hoje é um divã Família." Soniox and Gladia both invented the same emotional line in the Northeast clip, "eu adorei" (I loved it).

Flipped meaning. In the film clips, Gemini turned "you're on another team now" into "I'm on another team now" in the Rio scene. Besides the flip at the top of this post, the conversations gave us two more: "Não, imagina" (no way) became "Imagino" (I suppose so), and "E não acertei logo de início" (I didn't get it right at first) became "E acertei," dropping the negation, even though the very next words, "eu falhei" (I failed), contradict it. All three were in short spoken clauses, all three flipped a negation or a person, and none was invented from nothing.

Inserted content. In the TV interview, Gemini transcribed the music between segments as if it were speech, in three places, one of them a fragment of broken English. All three fell in gaps where ElevenLabs had output nothing. It didn't happen in the podcast, which has no music.

Dropped passages. In the Northeast clip, Gladia skipped the whole bakery exchange, about eight sentences.

Repetition loops. None recorded on these Portuguese clips.

One more error worth noting: Speechmatics turned a woman's "eu sou casada" (I'm married, feminine form) into the masculine "casado."

What we chose

Main track: ElevenLabs Scribe v2. The most complete on all three film scenes, with only minor inserted words.

Reference tracks:

• AssemblyAI (universal-2): overall the most accurate of the international engines. It invented one thing each in the São Paulo and Northeast clips, and nothing at all in the Rio clip, where it also had the best WER.
• Doubao: steady across all three Brazilian regions, with no invented sentences and no flips.
• Gemini: added on August 26, 2026.

AssemblyAI is listed ahead of Doubao in the configuration to match our other European languages. The order doesn't mean it's better.

Evidence against our choice

• Doubao is semantically safer than AssemblyAI. On the three film clips, Doubao invented nothing; AssemblyAI invented two things.
• Names are a blind spot for all three tracks. Names like João Guilherme and Dona Bárbara were wrong on all three tracks, each in a different way, so the merge step has nothing reliable to go on. Interviews with lots of names need the glossary.
• Gemini's three flips are real. In each case the surrounding lines show the contradiction. But these conversations were only compared against the main track; the other two reference tracks weren't run on them.
• As of our August records, there was no dedicated safeguard against background audio being written in as speech. A TV or radio playing at an interview location could end up in the transcript this way.

Open questions and limits

• There is no European Portuguese test clip. The only candidate film turned out to be a French production with mostly French dialogue, and the available sources were wrong anyway. European Portuguese films almost never have native subtitles, so we can't build a reference transcript. Our only European Portuguese evidence is the two conversations we added for Gemini, and they have no reference either.
• Doubao may not be reliable on European Portuguese. We send Doubao the Brazilian Portuguese language code. Our records say it quietly degrades into garble on European Portuguese, but without a test clip we have never verified this ourselves. There is one candidate fallback engine for European Portuguese, Speechmatics Melia-1, still waiting to be validated.
• All three film scenes are crowd scenes; there's no basic-tier clip.