Messages the client sends over the WebSocket connection.
Overview
| type | Transport | Purpose |
|---|---|---|
| input_audio_buffer.append | Text frame (JSON) | Send base64-encoded PCM audio |
| (binary frame) | Binary frame | Send raw PCM audio data |
| input_audio_buffer.flush | Text frame (JSON) | Trigger manual flush |
| flush | Text frame (JSON) | Alias for input_audio_buffer.flush |
| session.update | Text frame (JSON) | Update session parameters |
input_audio_buffer.append
Send base64-encoded PCM16LE audio data.
{
"type": "input_audio_buffer.append",
"audio": "<PCM16LE base64>"
}
| Field | Type | Required | Description |
|---|---|---|---|
| type | string | yes | Must be "input_audio_buffer.append" |
| audio | string | yes | Base64-encoded PCM16LE audio data |
Errors:
invalid_audio: audio is empty or not valid base64Binary audio frames
Send raw PCM binary data directly as an ArrayBuffer. Same effect as input_audio_buffer.append but without base64 encoding overhead.
Limits:
audio_frame_too_large and closes the connection (1009).concurrent_utterance_limit)input_audio_buffer.flush
Trigger a manual flush of the upstream buffer, ending the current speech segment.
{ "type": "input_audio_buffer.flush" }
"flush" is an alias with the same effect.
session.update
Update session parameters. Can be sent at any time after connection.
{
"type": "session.update",
"language": "zh",
"prompt": "Optional context hint"
}
Supported fields
| Field | Snake-case alias | Type | Description |
|---|---|---|---|
| language | — | string | Audio language (e.g. zh, en, auto) |
| prompt | — | string | Context hint, max 2000 characters |
| backend | — | string | Upstream backend selection |
| sttModel | stt_model | string | STT model |
| normalizerModel | normalizer_model | string | Text normalizer model |
| vad | — | boolean | Enable VAD |
| vadThreshold | vad_threshold | number | VAD sensitivity threshold |
| vadSilenceMs | vad_silence_ms | number | Silence duration to trigger end-of-speech |
| vadPrefixMs | vad_prefix_ms | number | Prefix padding duration |
| vadMinSpeechMs | vad_min_speech_ms | number | Minimum speech segment duration |
| vadTargetSpeechMs | vad_target_speech_ms | number | Target speech segment duration |
| vadMaxSpeechMs | vad_max_speech_ms | number | Maximum speech segment duration |
| vadSmartSplitWindowMs | vad_smart_split_window_ms | number | Smart split window |
::callout{icon="i-lucide-triangle-alert" color="amber"}
Tenant isolation
All fields not listed above are stripped before forwarding to the upstream. Identity and credential fields (request_id, session_id, user_id) are injected by the gateway and cannot be overridden by the client.
::
Errors
bad_json: JSON parse failurebad_message: Not a JSON object or unsupported typeinvalid_audio: Invalid audio data