Realtime Overview

Realtime is a Voice Context Layer mode for live speech. Recorded is the other mode, for full-file processing. See Voice Context Layer for the umbrella model.

The Realtime API provides streaming speech transcription over WebSocket. You send PCM16LE audio frames, the server returns transcription events.

What it does

Live captions: text appears while the person is still speaking

Real-time translation: source and translated text together (coming soon for external sessions)

Multi-language: Whisper-compatible language support

Server-side VAD: automatic speech segment detection, no client-side splitting needed

Input

Audio format: PCM16LE / 16kHz / mono

Transport: JSON text frames (base64) or binary frames (raw PCM)

Frame size: recommended 100ms frames (~3200 bytes), max 1 MiB

Output

Server pushes JSON events. The primary output is:

conversation.item.input_audio_transcription.completed — a speech segment has been transcribed

Supporting events:

input_audio_buffer.speech_started / speech_stopped — speech segment boundaries

session.created — connection established

error — error events

Architecture

Your app                   LansonAI Gateway                   Upstream STT
  │                            │                                  │
  ├── WS connect (Bearer) ───→ session.created                   │
  │                            │                                  │
  ├── Send PCM16 frames ──────→ gateway relay (binary PCM) ─────→ VAD + STT
  │                            │                                  │
   │←── input_audio_buffer.speech_started ──────│←── input_audio_buffer.speech_started
   │                            │                                  │
   │←── conversation.item.input_audio_transcription.completed ─│←── conversation.item.input_audio_transcription.completed
  │                            │                                  │
  ├── Close ─────────────────→ meter flush → R2 ledger            │

vs. Segment and Batch

Next steps

Realtime Quickstart — runnable example

Connection Lifecycle — connect, timeout, close

Audio Input — audio format reference