TechiSeek

VendorsFor Vendors

Cross-request KV cache exposure in the GLM-5.3 Workers AI model, and our remediation with SGLang

Cloudflare Blog

Cross-request KV cache exposure in the GLM-5.3 Workers AI model, and our remediation with SGLang News

Operators running GLM-5.3 on Workers AI should change two things: (1) size the prefill KV cache pool so it holds the full context window, and (2) patch the inference engine so any request carrying a pending rejection is retired at the disaggregation admission queue with its original error before bootstrap, allocation, or KV transfer occurs.

Implementation details

  • The fix was already on SGLang main in #36988, merged September 6 for a related failure path but not yet released; maintainers also merged #38935, which rejects these requests earlier, following Cloudflare's approach.
Visit Site

Recommended

What took the Internet offline in Q3 2026What took the Internet offline in Q3 2026Cloudflare Blog Cloudflare services for private origins is now generally availableCloudflare services for private origins is now generally availableCloudflare Blog Programming and Model RailroadsProgramming and Model RailroadsThenewstack nference Accelerates Self-Supervised Language Model Training with Cerebras CS-2 System - Cerebrasnference Accelerates Self-Supervised Language Model Training with Cerebras CS-2 System - CerebrasCerebras Cerebras Systems and Cirrascale Cloud Services® Introduce Cerebras AI Model Studio to TrainCerebras Systems and Cirrascale Cloud Services® Introduce Cerebras AI Model Studio to TrainCerebras Target Embraces Cross-Organizational DevOps CultureTarget Embraces Cross-Organizational DevOps CultureThenewstack New Spanish and Turkish Language Models and Updated General Models - Deepgram Blog ⚡️New Spanish and Turkish Language Models and Updated General Models - Deepgram Blog ⚡️Deepgram New Policy as Code Capabilities with CrossGuardNew Policy as Code Capabilities with CrossGuardPulumi The Go Memory Model - The Go Programming LanguageThe Go Memory Model - The Go Programming LanguageGo What is a CUDA Thread Block?What is a CUDA Thread Block?Modal Docker Desktop 4.23: Updates to Docker Init, New Configuration Integrity Check, Quick SearchDocker Desktop 4.23: Updates to Docker Init, New Configuration Integrity Check, Quick SearchDocker Bringing More Power To Edge Rules! Introducing Variable ExpansionBringing More Power To Edge Rules! Introducing Variable ExpansionBunny INTERNAL_STATIC_REQUEST_FAILEDINTERNAL_STATIC_REQUEST_FAILEDVercel NO_EXTERNAL_CSS_AT_IMPORTSNO_EXTERNAL_CSS_AT_IMPORTSVercel You can just ship agentsYou can just ship agentsVercel Fields - MintlifyFields - MintlifyMintlify Speech-to-Text API model for Pharma Use Cases | Nova-3 PharmaSpeech-to-Text API model for Pharma Use Cases | Nova-3 PharmaDeepgram What You Need To Know About OpenAI’s New 3D Model-Making AI, Point-E - Deepgram Blog ⚡️What You Need To Know About OpenAI’s New 3D Model-Making AI, Point-E - Deepgram Blog ⚡️Deepgram Strong Security Doesn’t Have to Equate to Slow DevelopmentStrong Security Doesn’t Have to Equate to Slow DevelopmentThenewstack Bitbucket Cloud Meets Pulumi CloudBitbucket Cloud Meets Pulumi CloudPulumi