Client Messages

Messages the client sends over the WebSocket connection.

Overview

type	Transport	Purpose
`input_audio_buffer.append`	Text frame (JSON)	Send base64-encoded PCM audio
(binary frame)	Binary frame	Send raw PCM audio data
`input_audio_buffer.flush`	Text frame (JSON)	Trigger manual flush
`flush`	Text frame (JSON)	Alias for `input_audio_buffer.flush`
`session.update`	Text frame (JSON)	Update session parameters

input_audio_buffer.append

Send base64-encoded PCM16LE audio data.

{
  "type": "input_audio_buffer.append",
  "audio": "<PCM16LE base64>"
}

Field	Type	Required	Description
`type`	string	yes	Must be `"input_audio_buffer.append"`
`audio`	string	yes	Base64-encoded PCM16LE audio data

Errors:

invalid_audio: audio is empty or not valid base64

Binary audio frames

Send raw PCM binary data directly as an ArrayBuffer. Same effect as input_audio_buffer.append but without base64 encoding overhead.

Limits:

Max frame size: 1 MiB (1,048,576 bytes). Exceeding this returns audio_frame_too_large and closes the connection (1009).

Frames are dropped when concurrent utterance limit is exceeded (concurrent_utterance_limit)

Frames are dropped or connection closed (1013) on upstream backpressure

input_audio_buffer.flush

Trigger a manual flush of the upstream buffer, ending the current speech segment.

{ "type": "input_audio_buffer.flush" }

"flush" is an alias with the same effect.

session.update

Update session parameters. Can be sent at any time after connection.

{
  "type": "session.update",
  "language": "zh",
  "prompt": "Optional context hint"
}

Supported fields

Field	Snake-case alias	Type	Description
`language`	—	string	Audio language (e.g. `zh`, `en`, `auto`)
`prompt`	—	string	Context hint, max 2000 characters
`backend`	—	string	Upstream backend selection
`sttModel`	`stt_model`	string	STT model
`normalizerModel`	`normalizer_model`	string	Text normalizer model
`vad`	—	boolean	Enable VAD
`vadThreshold`	`vad_threshold`	number	VAD sensitivity threshold
`vadSilenceMs`	`vad_silence_ms`	number	Silence duration to trigger end-of-speech
`vadPrefixMs`	`vad_prefix_ms`	number	Prefix padding duration
`vadMinSpeechMs`	`vad_min_speech_ms`	number	Minimum speech segment duration
`vadTargetSpeechMs`	`vad_target_speech_ms`	number	Target speech segment duration
`vadMaxSpeechMs`	`vad_max_speech_ms`	number	Maximum speech segment duration
`vadSmartSplitWindowMs`	`vad_smart_split_window_ms`	number	Smart split window

::callout{icon="i-lucide-triangle-alert" color="amber"} Tenant isolation All fields not listed above are stripped before forwarding to the upstream. Identity and credential fields (request_id, session_id, user_id) are injected by the gateway and cannot be overridden by the client. ::

Errors

bad_json: JSON parse failure

bad_message: Not a JSON object or unsupported type

invalid_audio: Invalid audio data

Server Events — server-pushed events

Session Configuration — session config details

Audio Input — audio format reference