[{"data":1,"prerenderedAt":4},["ShallowReactive",2],{"article-doc:docs\u002Frealtime\u002Findex":3},"---\ntitle: Realtime Overview\ndescription: What the Realtime API does, input, output, and architecture.\nstatus: stable\n---\n\n# Realtime Overview\n\nRealtime is a **Voice Context Layer** mode for live speech. [Recorded](\u002Fdocs\u002Frecorded) is the other mode, for full-file processing. See [Voice Context Layer](\u002Fdocs\u002Fconcepts) for the umbrella model.\n\nThe Realtime API provides streaming speech transcription over WebSocket. You send PCM16LE audio frames, the server returns transcription events.\n\n## What it does\n\n- **Live captions**: text appears while the person is still speaking\n- **Real-time translation**: source and translated text together *(coming soon for external sessions)*\n- **Multi-language**: Whisper-compatible language support\n- **Server-side VAD**: automatic speech segment detection, no client-side splitting needed\n\n## Input\n\n- Audio format: **PCM16LE \u002F 16kHz \u002F mono**\n- Transport: JSON text frames (base64) or binary frames (raw PCM)\n- Frame size: recommended 100ms frames (~3200 bytes), max 1 MiB\n\n## Output\n\nServer pushes JSON events. The primary output is:\n\n- `conversation.item.input_audio_transcription.completed` — a speech segment has been transcribed\n\nSupporting events:\n\n- `input_audio_buffer.speech_started` \u002F `speech_stopped` — speech segment boundaries\n- `session.created` — connection established\n- `error` — error events\n\n## Architecture\n\n```\nYour app                   LansonAI Gateway                   Upstream STT\n  │                            │                                  │\n  ├── WS connect (Bearer) ───→ session.created                   │\n  │                            │                                  │\n  ├── Send PCM16 frames ──────→ gateway relay (binary PCM) ─────→ VAD + STT\n  │                            │                                  │\n   │←── input_audio_buffer.speech_started ──────│←── input_audio_buffer.speech_started\n   │                            │                                  │\n   │←── conversation.item.input_audio_transcription.completed ─│←── conversation.item.input_audio_transcription.completed\n  │                            │                                  │\n  ├── Close ─────────────────→ meter flush → R2 ledger            │\n```\n\n## vs. Segment and Batch\n\n| Dimension | Realtime WS | Segment HTTP | Batch jobs |\n|---|---|---|---|\n| Transport | WebSocket | HTTP sync | HTTP async |\n| Latency | Milliseconds | ~≤1.6s | Minutes |\n| Input | PCM16LE stream | Multipart `file` | Audio URL |\n| Output | Event stream | Transcript JSON | Complete job JSON |\n| VAD | **Server** | **Client** | **Server** (slice) |\n| Best for | Live stream | Pre-cut utterances | Long files |\n\n## Next steps\n\n- [Realtime Quickstart](\u002Fdocs\u002Frealtime\u002Fquickstart) — runnable example\n- [Connection Lifecycle](\u002Fdocs\u002Frealtime\u002Fconnection-lifecycle) — connect, timeout, close\n- [Audio Input](\u002Fdocs\u002Frealtime\u002Faudio-input) — audio format reference\n",1790059118958]