Cloudflare has admitted that, for almost two weeks, a fault in one of its hosted AI models sent fragments of some customers’ requests to other customers. The cause was a capacity misconfiguration that ran into a bug in the open-source SGLang inference engine, and anyone running SGLang themselves has patching to do.
What happened
On 26 October 2026 Cloudflare published a detailed post-incident report on a tenant isolation failure affecting the GLM-5.3 model (@cf/zai-org/glm-5.3) on its Workers AI platform. Between 27 August, the day the model went live, and 9 September 2026, a total of 155 responses came back containing material from requests other customers had submitted shortly before. Every affected response came from an unusually long prompt, above 681,280 tokens.
The problem was found by Gadi Evron of Knostic, who reported it through Cloudflare’s HackerOne programme on 8 September. Cloudflare began verifying the report the next morning, declared an incident at 13:22 UTC and reproduced the leak from two separate accounts within 25 minutes. The model was made private at 14:00, the root cause was pinned down by 14:35, and a corrected build went back into public service at 19:40 that evening.
Cloudflare says the affected requests looked like normal long-context work in timing and volume, and it found no sign that anyone had deliberately exploited the flaw. It also says no other Workers AI model was affected. SGLang’s maintainers published their own advisory the same day. The researcher’s write-up is still to come.
The technical picture
Model servers keep a key-value (KV) cache of attention data computed from your prompt, so it is not recalculated for every generated token. In a shared service those cache pages are recycled between customers constantly, and isolation depends on no request ever reading pages still holding someone else’s data.
Two problems lined up. First, GLM-5.3 was advertised with a context window of 1,048,576 tokens, but the prefill KV pool on Cloudflare’s nodes could hold only 681,280. Cloudflare says it knew about the gap before launch and assumed the engine would simply reject anything too large. Its long-context testing looked at HTTP status codes and performance, not at whether the output was actually correct, so the assumption was never tested.
Second, SGLang did not behave as expected in its disaggregated mode, where one set of servers handles prefill (reading the prompt) and another handles decode (generating the answer). An over-length request was flagged as rejected but still went through to decode as a one-token placeholder. The decode node then attended over stale cache pages left behind by earlier requests and returned what it produced with an HTTP 200, instead of the error a single-node deployment would have sent back. Cloudflare’s report notes that another SGLang user had already raised the underlying behaviour as an issue on 6 July.
SGLang’s advisory makes clear that this is wider than Cloudflare. Default disaggregated configurations using RDMA-based transfer backends (Mooncake, NIXL, MORI or a custom backend) were exposed to a related race from the v0.4.x line in early 2025 until fixes landed this month. Single-instance deployments without the cross-engine transfer path are not affected.
Why it matters for UK organisations
Many UK firms now send contracts, customer records and source code to hosted models, often via a developer’s API key rather than formal procurement. Prompt confidentiality relies on cache management deep inside someone else’s inference stack, not just on encryption and access controls.
There is a data protection side too. If personal data was in any of the leaked fragments, the customers whose prompts were exposed are controllers with UK GDPR duties, and whether a reportable breach occurred will depend on what was in those prompts. Workers AI users who ran very long GLM-5.3 prompts between 27 August and 9 September should ask Cloudflare directly whether their account was involved, and record that assessment whatever the answer.
The second group is the growing number of UK organisations self-hosting open-weight models to keep data in-house. SGLang is one of the most popular serving engines for that job. If you run it in disaggregated mode on an older build, you may have the same exposure between your own internal tenants, business units or customers.
Expert view
The detail that stands out to me is the testing gap. Cloudflare checked that oversized requests produced the right status code and stopped there. In my experience that is exactly how isolation failures slip into production: functional tests prove the happy path, nobody asserts that the response actually belongs to the caller, and an unhandled edge case becomes a cross-customer leak. When we test multi-tenant platforms we plant canary values in one tenant and hunt for them in another; AI inference deserves the same, before a researcher finds it.
Credit where it is due: Cloudflare took the model offline within about 40 minutes of declaring the incident, worked with the upstream maintainers, and published unusually detailed root cause analysis, while SGLang listed affected versions and backends plainly. That is coordinated disclosure done properly.
The broader lesson is that AI serving stacks are now critical shared infrastructure with complex memory sharing and remote memory transfer, and they have not had decades of hardening like hypervisors or database engines. Cloudflare is working on per-page integrity tags and formal verification of its transfer protocol; expect more bugs of this kind before such measures become normal.
What to do now
- Find out whether you were exposed on Workers AI. Check logs for GLM-5.3 calls with very long prompts between 27 August and 9 September 2026, ask Cloudflare whether your account was affected, and run a UK GDPR breach assessment if personal data could have been involved.
- Upgrade self-hosted SGLang. If you run disaggregated prefill/decode, move to v0.5.22 or later; MORI users need at least that build, and SGLang says a further fix for failed NIXL and MORI sends is due in v0.5.23. Fixes have not been backported to older branches. This is Cyber Essentials security update management applied to AI infrastructure.
- Check capacity against advertised context. Confirm on every prefill and decode node that the KV pool can hold the full context length, and recheck after any change to weights, quantisation, parallelism or memory settings. Treat it as a secure configuration item.
- Turn on the integrity controls. Enable deferred KV release on both roles and consider SGLang’s opt-in transfer checksum. Make sure load balancers do not cancel in-flight transfers when a client drops its connection.
- Test for cross-tenant leakage. Add canary-based tests to AI platform release pipelines and pentest scopes, checking content as well as status codes.
- Classify what goes into hosted models. Restrict which data classes staff and applications may send to third-party inference, and make sure AI providers appear in your supplier risk register with their incident notification terms understood.
Bottom line
This was not a sophisticated attack, and that is what makes it worth noting. A known sizing mismatch, an untested assumption and an engine bug were enough for a major provider to hand other customers’ data out for 13 days. Workers AI users should confirm their exposure, SGLang operators should patch this week, and AI buyers should demand the isolation guarantees they expect of any multi-tenant platform.
