Realtime Translation

Real-time translation: get translated text alongside transcription while speech is happening.

::callout{icon="i-lucide-triangle-alert" color="amber"} Real-time translate_to is not currently enabled for external sessions. The public realtime gateway forwards language but does not forward translate_to, so a session.update with that field has no effect. ::

Usage

Configure translation

Set the source language and translation target via session.update:

{
  "type": "session.update",
  "language": "zh",
  "translate_to": "en"
}

language: source language of the audio

translate_to: target language for translation

Translation output

When enabled, translation results are returned alongside transcription in the conversation.item.input_audio_transcription.completed event.

Source and translated text correspondence

Source and translation for each utterance arrive in the same conversation.item.input_audio_transcription.completed event, ensuring correspondence. No client-side matching needed.

Switching languages

Change language at any time during the session:

{ "type": "session.update", "language": "en" }

New utterances after the switch use the new language. In-flight utterances complete with the previous language.

Latency

Translation adds additional latency:

Translation latency depends on audio length, context, and service load. Measured data on Service Status.

Multilingual / code-switching

Auto-detect: set language: "auto"

Mixed language: upstream supports code-switching, accuracy depends on model and audio quality

Chinese aliases: zh-cn, zh-tw, mandarin etc. are normalized to zh

UI strategy

1. Receive conversation.item.input_audio_transcription.completed event 2. Display source text on first line 3. Display translation on second line 4. Each utterance is independent, no replacement needed

See Build Live Translation for a detailed guide.

Related

Session Configuration — session.update details

Live Translation — translation concepts

Build Live Translation — UI guide