A language detector always gives an answer, even to a cough. Leave detection on for clean or mixed-language audio, and name the language when one language is spoken over noise or a heavy accent.
Every transcription starts with a question the reader never sees: what language is this? Get it wrong and the rest of the pipeline works hard at producing fluent nonsense. This article explains what a language detector listens for, how detection behaves for live audio and for files, and when you are better off naming the language yourself.
A language detector does not understand what is being said. It looks at the sound itself: the inventory of consonants and vowels, how syllables are stressed, the rhythm of the phrases, and the short, frequent words that every language leans on. Spanish has its own vowel set and rhythm; Dutch and German share a great deal but part ways on a handful of sounds. A few seconds of clear speech is usually enough to tell them apart.
Many recognisers do this as a by-product of recognising the words. They decide the language from the opening of the audio, as a probability for every language they know. Two things follow from that design. The probabilities always add up to one, so there is no "no language" answer: silence, a cough or a music sting gets a language too, often with a confident score attached. And the decision is made from the opening, which is the worst sample you could pick. Many recordings open with greetings, names and place names, which carry very little language signal: "Good morning, Maria, welcome to Berlin" is nearly as German as it is English. Others open with silence or a cough, and a detector asked to decide on that will decide anyway, with confidence, on nothing at all. Real audio does this too: a Romanian sermon that opens with room noise and "Bună dimineața" can easily be heard as Italian "Buona domenica", with complete certainty.
The cost of a wrong answer is also worse than it sounds. A recogniser decoding in the wrong language does not misspell; it rewrites. "For God so loved the world", decoded as Romanian, came out as "Forgot to love the world". The Romanian "mândria slujește", decoded as English, came out as "Pride serves", which is an accurate translation of a sentence nobody asked to have translated. The words are fluent and the transcript gives no sign that anything is wrong, which is what makes the error expensive.
Five things fool detectors most often:
Which of these bites you depends on whether the audio is arriving live or already sitting in a file.
For a live transcription, there is no whole recording to look at. The audio arrives as it is spoken, and the transcript has to arrive with it. So the language is detected as the speaker talks, and followed if they switch. This is what the note under the Spoken language switch means when you start a live transcription in the Dashboard: the speaker's language is detected automatically, and they may switch at any time.
A switch is confirmed before it is applied, so one noisy patch does not drag the transcript into another language. The weak spot is the opening: if the first words are a noisy greeting, the first line may come out in the wrong language. Naming the language avoids that.
Files work differently, because the whole recording is already there.
A file is a better problem: there is no need to trust the first seconds. For a file, the language is decided from the recording as a whole, so a noisy greeting at the start cannot settle it.
The file's language is still a working assumption, not a verdict. A recording that genuinely changes language partway, such as an interview that moves from English into Romanian for its second half, is followed rather than forced, and one that alternates between languages throughout is handled too.
A detector that decides once and never looks back is confident in the wrong place.
No detector helps much when the whole recording is noisy. Detection is the default because it is right most of the time and costs you nothing. For some recordings, though, you should name the language, and the Dashboard says which ones: naming the spoken language improves accuracy on noisy recordings and heavy accents.
Those are the cases where detection has the least to work with. Noise removes the cues a detector depends on; a heavy accent shifts them towards another language; a closely related pair offers few cues to begin with. An English speaker with a strong Portuguese accent in a reverberant hall is hard to place from the sound alone, and if the detector places them wrong, the transcript turns English speech into Portuguese for as long as the mistake lasts. Naming the language removes the question, so all of the effort goes into the words.
A short checklist:
Naming the language is a setting of the transcription, not of your account, so you can decide per recording. Everything above comes down to one API option.
spokenLanguage absent means detect; session.update mid-streamIn the API, spokenLanguage is a language code such as en; leave it out and the language is detected. It appears in the settings of a file or link transcription and in the options of a realtime session. Detection and naming cover the same 100 languages. Translation into up to 10 of them is a separate choice, and it does not depend on which language was detected or named. How translation keeps pace with speech is the subject of Live translation latency, explained.
{ "name": "Board meeting", "settings": { "spokenLanguage": "en", "speakerLabels": "auto" } }
A realtime session takes the same option when it is created, and you can change it while audio is flowing. A session.update text frame with a new spokenLanguage applies to speech from then on; setting it to "" or "auto" returns the session to detection, and an unsupported code is ignored. A caption view with a language button can let the operator name the language the moment they see a wrong first line, without reconnecting.
{ "type": "session.update", "spokenLanguage": "ro" }
The list of codes, with their names, is at GET /v1/languages, which needs no key. The full realtime flow, from creating a session on your server to reading drafts and commits, is in Real-time transcription over a WebSocket, and the interactive reference is at {{api}}/docs. In the Dashboard, the same choice is the Spoken language switch when you start a new transcription, and keys for the API live under Developers.