Streaming LLM Side Channels: Token Length & Timing (2026)

Q: What is a token-length side channel in streaming LLMs?

It is a leak of information through the sizes of encrypted network records rather than their contents. When an LLM response is streamed, each token or small group of tokens is usually sent as its own server-sent event inside its own TLS record. TLS hides the bytes but not the record length, and the envelope around each delta is nearly constant, so an observer can subtract it and recover the length of each token. A sequence of token lengths constrains which words were sent. Weiss and colleagues showed in 2024 that this was enough to reconstruct 27 percent of assistant responses accurately and infer the topic of 53 percent, using a model trained to turn length sequences back into text.

Q: What is Whisper Leak and what did it show?

Whisper Leak is research published by Microsoft in November 2025 by Geoff McDonald and Jonathan Bar Or. It showed that an observer of encrypted streaming LLM traffic can classify whether a conversation is about a specific sensitive topic using packet sizes and inter-arrival times. Across 28 popular LLMs from major providers, their classifiers often exceeded 98 percent area under the precision-recall curve, and in a realistic setting with one target conversation among many thousands of unrelated ones they reported 100 percent precision for some topics while still catching a meaningful fraction of target conversations. They also tested random padding, token batching and packet injection and found each reduced effectiveness without eliminating it.

Q: Does TLS encryption not protect LLM conversations?

TLS protects the content of each record against reading and tampering, which is what it was designed to do. It does not hide metadata such as how many records were sent, how large each one was and when it arrived. For most web traffic that metadata reveals little. A streaming LLM response is unusual because it sends many tiny records whose sizes and timing are tightly coupled to the text being generated, which turns ordinary metadata into a usable signal. So TLS remains essential, but it is not sufficient for streamed AI responses on sensitive topics; you also need to control the shape of the traffic on each hop you own.

Q: If my LLM provider already pads its stream, am I protected?

Only on the hop the provider controls. Cloudflare added random padding to Workers AI and AI Gateway in 2024, and the OpenAI SDK documents an obfuscation field on streaming deltas that is enabled by default. That protects the connection between the provider and whoever terminates it, usually your backend or gateway. Most products then parse the stream and forward each text delta to the browser as a new event, discarding the padding. The hop from your service to the user, which crosses Wi-Fi, ISPs and corporate networks, is then unpadded and per-token again. You need to apply your own shaping on every hop that re-emits the stream.

Q: What is the most cost-effective defence for a typical chat product?

Batching with bucketed padding at the backend-for-frontend that serves the browser. Accumulate model output and emit it on a timer, for example every 100 to 200 milliseconds or every 64 bytes, whichever comes first, and pad each event up to a fixed size bucket such as 128, 256 or 512 bytes. Users still see a smooth stream because people cannot read faster than that cadence, while the network sees coarse windows rather than individual token lengths, and timer-driven emission blurs generation timing. It is a small change in one service and removes most of the per-token signal, though total size and duration still leak.

Q: When should I stop streaming responses altogether?

When the topic of the conversation is itself sensitive and the users may be on networks you do not trust: health and mental health assistants, legal advice, financial hardship, HR complaints, or anything a user would not say aloud in public. For those tiers, either return a non-streamed response padded into a size bucket, or stream at a constant rate with fixed-size frames, dummy frames when idle, and a random tail after completion. Both cost latency or bandwidth. For low-sensitivity features such as public documentation bots or internal coding assistants over a corporate VPN, streaming with batching is fine and stronger measures are not worth the cost.

Q: Do timing side channels matter if I pad every event?

Yes. Padding hides sizes but not when records are sent. Carlini and Nasr showed that efficiency techniques such as speculative decoding make generation speed depend on the content, with more than 90 percent precision at distinguishing medical from coding conversations on open-source systems and distinguishable messages on production services. If you forward each delta the moment it arrives, the timing of your events mirrors the model. Emitting on a fixed timer decouples event timing from generation within each window, and constant-rate emission with dummy frames removes it, apart from total duration. If you run your own inference, decide per sensitivity tier whether speculative decoding is worth its timing signal.

Q: How do I test whether my defences actually work?

Attack yourself. Capture your own production-like traffic on the client-facing hop, label conversations by topic using synthetic prompts, and train a simple classifier such as gradient-boosted trees on event sizes and inter-arrival times, which is the same approach the published research used. Measure its precision and recall before and after each change, and keep the measurement as a recurring security metric rather than a one-off test. If the classifier still reliably identifies your sensitive topics, the defence is not working regardless of what the configuration says. Make sure the captured traffic itself is stored under strict access control, because it is exactly the dataset an attacker would want.

Satyam Kumar

返回博客

cyber-security-patterns

Encryption Hides the Words, Not the Rhythm: Token-Length and Timing Side Channels in Streaming LLMs, and Why Your Own Proxy Is the Hop That Leaks (2026)

By Satyam KumarOctober 1, 202612 min read

llm side channel attack whisper leak token length side channel streaming llm security encrypted traffic analysis server-sent events security speculative decoding timing attack openai stream obfuscation cloudflare ai gateway padding traffic padding defence llm gateway security backend for frontend streaming owasp llm sensitive information disclosure ai privacy engineering network metadata leakage llm security architecture

Encryption Hides the Words, Not the Rhythm: Token-Length and Timing Side Channels in Streaming LLMs, and Why Your Own Proxy Is the Hop That Leaks (2026)

Frequently Asked Questions

分享这篇文章

Twitter LinkedIn WhatsApp

Satyam Kumar

Founder & AI Architect, AppScale LLP

人工智能和云架构师。帮助团队构建可扩展到数百万的系统。

LinkedIn GitHub

Encryption Hides the Words, Not the Rhythm: Token-Length and Timing Side Channels in Streaming LLMs, and Why Your Own Proxy Is the Hop That Leaks (2026)

Frequently Asked Questions

分享这篇文章

Comments

Leave a comment