How to build realtime captions that don't jitter
> TL;DR — Most live-caption tutorials assume you receive a stream of partial text that you must keep replacing until it "finalizes." That model is what causes the flicker. LansonAI's realtime API takes the opposite approach: it only emits stable text. Your client becomes append-only — no replacement mapping, no rollback, no debounce. This post shows the model and the minimal code.
Problem
You build a live caption UI. The first demo looks fine. Then a real speaker talks at normal speed, and the captions start to misbehave:
The result is worse than no captions at all. Users stop trusting what they're reading, because they can't tell what's final and what's still being decided.
The usual fix is to build a partial → final state machine: keep the latest partial in a "working" slot, and only commit it to the transcript when the API tells you it's final. You add replacement logic, rollback logic, and debounce to hide the churn. It works, but it's a lot of client complexity for a problem that's really a protocol problem.
Architecture
The root cause is a design choice about what the server sends.
There are two models:
Model 1 — Partial stream (the common one). The server sends a best-guess on every audio chunk. The client renders the guess, then keeps overwriting it as better guesses arrive, until a "final" event locks it in. The client owns the stability logic.
server: "hel" → "hello" → "hello w" → "hello wo" → FINAL "hello world"
client: renders each, replaces the previous, commits on FINAL
Model 2 — Stable stream (LansonAI's StableStream). The server does the stabilization work. It only sends an event when an utterance has reached a stable boundary. What it sends is final. The client never sees a partial, so there's nothing to replace.
server: (silence while the utterance is still forming)
→ FINAL "hello world"
client: appends. done.
The key reframe: stability is not a binary wrong → correct. It's a lifecycle — mutable → increasingly reliable. Model 1 pushes the whole lifecycle to the client. Model 2 collapses it server-side and hands the client only the stable end.
> Why we built it this way — In live speech, the system doesn't have all the information until the utterance is over. Sending partials is cheap, but it exports the hardest part of the problem (deciding what's stable) to every client, in every language, in every framework. Doing it once, server-side, means every consumer gets the same stable contract. This is the same insight behind our "accuracy is not the same as readable" position: a caption that is 99% accurate but keeps rewriting itself is less usable than one that is 97% accurate and never moves.
Code
The full audio pipeline (mic capture → 16kHz mono PCM16LE → 100ms frames → WebSocket) is covered in [How to stream microphone audio over WebSocket](/cookbook/stream-mic-audio-over-websocket). Here we focus on the render side, which is where jitter is born and killed.
1. Connect (browser)
Browser connections use a short-lived session token from your backend (never put your secret key in the browser):
// 1. Get a session token from your backend
const { token } = await fetch("/your-backend/session-token", {
method: "POST",
}).then((r) => r.json());
// 2. Open the realtime stream
const ws = new WebSocket(
wss://audio.lansonai.com/v1/audio/transcriptions/stream?access_token=${token}
);
2. Append-only render (the whole trick)
Because every completed event is already final, the render path is a plain append. No working slot, no replacement, no rollback:
const captions = document.getElementById("captions");
function appendCaption(utteranceIndex, text) {
const p = document.createElement("p");
p.dataset.utterance = utteranceIndex;
p.textContent = text;
// Auto-scroll only if the user is already near the bottom
const nearBottom =
captions.scrollHeight - captions.scrollTop - captions.clientHeight < 100;
captions.appendChild(p);
if (nearBottom) captions.scrollTop = captions.scrollHeight;
}
ws.onmessage = (event) => {
const data = JSON.parse(event.data);
// Optional: lightweight status indicator while an utterance is in flight
if (data.type === "input_audio_buffer.speech_started") {
showIndicator(data.utterance_index, "listening");
}
if (data.type === "input_audio_buffer.speech_stopped") {
showIndicator(data.utterance_index, "processing");
}
// The only event that changes the transcript:
if (data.type === "conversation.item.input_audio_transcription.completed") {
hideIndicator(data.utterance_index);
appendCaption(data.utterance_index, data.text); // final, stable text
}
};
3. What you do NOT need to write
This is the part that surprises people coming from a partial-stream API:
The transcript is a monotonically growing list. Each completed event appends one immutable line.
Result
The trade you're making — waiting for stable instead of rendering partials — is bounded, not open-ended. In our own flood testing, utterance completion landed around 500–650ms p50; production traffic runs a heavier tail. Notice what that variance does not affect: your client code. Because committed text never moves, latency only changes when a line appears — never whether what the reader already saw is still true.
Why it matters
1. Stability is a protocol decision, not a UI trick. You can paper over jitter with debounce and replacement logic, but you're re-implementing, per client, a decision that belongs in one place. Push it server-side and every consumer inherits it for free.
2. Accuracy and readability are different axes. A high-accuracy caption that keeps rewriting itself reads as broken, not as smart. Reflow is a comprehension cost: when a line moves, the reader has to relocate it and re-decide whether what they already read still means the same thing — effort that competes directly with listening. Users calibrate trust on stability, not on WER. (This is the engineering version of our "readable captions beyond accuracy" argument.)
3. The transferable lesson. Whenever you stream a provisional result (transcription, translation, LLM output, search), decide explicitly where stabilization happens. If you send partials, you've handed your hardest problem to every client. If you stabilize once and emit only the stable result, your clients get simpler and your contract gets stronger.