[{"data":1,"prerenderedAt":4},["ShallowReactive",2],{"article-doc:docs\u002Frealtime\u002F4.audio-input":3},"---\ntitle: Audio Input\ndescription: PCM format, sample rate, channels, chunk size, and latency tradeoffs.\n---\nThe realtime API accepts PCM16LE audio. This page covers format requirements, frame size, and latency tradeoffs.\n\n## Format requirements\n\n| Property | Requirement |\n|---|---|\n| Encoding | PCM16LE (16-bit signed little-endian) |\n| Sample rate | 16000 Hz |\n| Channels | 1 (mono) |\n\n::callout{icon=\"i-lucide-triangle-alert\" color=\"amber\"}\nAudio not matching this format will be rejected or produce incorrect results. Convert with ffmpeg first.\n::\n\n## Converting audio\n\n```bash\nffmpeg -i input.mp3 -ar 16000 -ac 1 -c:a pcm_s16le output.wav\n```\n\n## Transport methods\n\n### Text frames (base64)\n\n```json\n{ \"type\": \"input_audio_buffer.append\", \"audio\": \"\u003CPCM16LE base64>\" }\n```\n\nHas ~33% base64 overhead. Good for debugging and quick integration.\n\n### Binary frames (raw PCM)\n\nSend raw `ArrayBuffer` directly. No encoding overhead. Recommended for production.\n\n## Frame size and latency\n\n| Frame size | Interval | Data size | Latency impact |\n|---|---|---|---|\n| 50ms | 50ms | 1600 bytes | Lowest latency, higher CPU overhead |\n| **100ms** | **100ms** | **3200 bytes** | **Recommended balance** |\n| 200ms | 200ms | 6400 bytes | Increased latency, fewer frames |\n| 500ms | 500ms | 16000 bytes | Noticeable latency |\n\n**100ms frames** (3200 bytes) is the recommended sweet spot.\n\n## Max frame size\n\n1 MiB (1,048,576 bytes). Exceeding this returns `audio_frame_too_large` and closes the connection (1009).\n\n## Browser capture\n\n```javascript\nconst ctx = new AudioContext({ sampleRate: 16000 });\nconst stream = await navigator.mediaDevices.getUserMedia({ audio: true });\nconst source = ctx.createMediaStreamSource(stream);\n\u002F\u002F Use AudioWorklet or ScriptProcessorNode to convert float samples to PCM16LE\n\u002F\u002F Ensure 16000 Hz sample rate and mono\n```\n\n## Server-side capture\n\n- Use `Authorization: Bearer sk-...` header\n- Send binary PCM frames directly (no base64 needed)\n- Keep sending to avoid idle timeout\n\n## Backpressure\n\nWhen the upstream cannot keep up:\n\n| Condition | Behavior |\n|---|---|\n| Soft threshold (1 MiB in-flight) | Drop frames, send `lanson.throttled` |\n| Hard threshold (8 MiB in-flight) | Close connection (1013) |\n\nMonitor `lanson.throttled` events and reduce send rate accordingly.\n\n## Related\n\n- [Client Messages](\u002Fdocs\u002Fapi-reference\u002Fclient-messages) — audio message format\n- [Connection Lifecycle](\u002Fdocs\u002Frealtime\u002Fconnection-lifecycle) — idle timeout\n- [Realtime Quickstart](\u002Fdocs\u002Frealtime\u002Fquickstart) — complete example\n",1790059118957]