Realtime Overview

Realtime is a Voice Context Layer mode for live speech. [Recorded](/docs/recorded) is the other mode, for full-file processing. See [Voice Context Layer](/docs/concepts) for the umbrella model.

The Realtime API provides streaming speech transcription over WebSocket. You send PCM16LE audio frames, the server returns transcription events.

What it does

Live captions: text appears while the person is still speaking

Real-time translation: source and translated text together (coming soon for external sessions)

Multi-language: Whisper-compatible language support

Server-side VAD: automatic speech segment detection, no client-side splitting needed

Input

Audio format: PCM16LE / 16kHz / mono

Transport: JSON text frames (base64) or binary frames (raw PCM)

Frame size: recommended 100ms frames (~3200 bytes), max 1 MiB

Output

Server pushes JSON events. The primary output is:

conversation.item.input_audio_transcription.completed — a speech segment has been transcribed

Supporting events:

input_audio_buffer.speech_started / speech_stopped — speech segment boundaries

session.created — connection established

error — error events

Architecture

Your app                   LansonAI Gateway                   Upstream STT
  │                            │                                  │
  ├── WS connect (Bearer) ───→ session.created                   │
  │                            │                                  │
  ├── Send PCM16 frames ──────→ gateway relay (binary PCM) ─────→ VAD + STT
  │                            │                                  │
   │←── input_audio_buffer.speech_started ──────│←── input_audio_buffer.speech_started
   │                            │                                  │
   │←── conversation.item.input_audio_transcription.completed ─│←── conversation.item.input_audio_transcription.completed
  │                            │                                  │
  ├── Close ─────────────────→ meter flush → R2 ledger            │

vs. Segment and Batch

| Dimension | Realtime WS | Segment HTTP | Batch jobs | |---|---|---|---| | Transport | WebSocket | HTTP sync | HTTP async | | Latency | Milliseconds | ~≤1.6s | Minutes | | Input | PCM16LE stream | Multipart file | Audio URL | | Output | Event stream | Transcript JSON | Complete job JSON | | VAD | Server | Client | Server (slice) | | Best for | Live stream | Pre-cut utterances | Long files |

Next steps

[Realtime Quickstart](/docs/realtime/quickstart) — runnable example

[Connection Lifecycle](/docs/realtime/connection-lifecycle) — connect, timeout, close

[Audio Input](/docs/realtime/audio-input) — audio format reference