Cancellation Is Not Enough: What Retries Leave Behind in an Overloaded LLM Engine
arXiv:2611.08684v1 Announce Type: new Abstract: Clients and proxies in front of an LLM engine retry calls that time out. After a transient overload, these retries can keep the engine from recovering even when calls arrive below its capacity. Since the engine may keep computing for clients that have left,…
Excerpt provided by arXiv cs.AI via its public feed. Full reporting belongs to the original publisher.
Read the full story on arXiv cs.AI
Related coverage
The Decoder · 31m ago
Gemini 4 Carbon arrives just in time to feel like yesterdays news in the fastest model cycle yet
Ars Technica AI · 3h ago
DeepMind reveals Gemini Robotics 2.5 and open safety benchmark "ASIMOV-jailbreak"
Google DeepMind · 6h ago
Introducing AlphaProof Code: a breakthrough in verifying complex software at depth
OpenAI · 9h ago
Closing the AI guidance gap in education