Multi-GPU MoE Inference with Adaptive Expert Offloading
arXiv cs.AI · William X. Zheng, Zhongjie Chen, Shuo Yang, Ziming Mao, Xiaoze Fan, Yi Pan, Kunjun Li, Hanfei Yu, Chenfeng Xu, Yang Zhou, Rishabh Iyer, Ion Stoica ·
arXiv:2611.10219v1 Announce Type: new Abstract: Modern Mixture-of-Experts (MoE) inference is constrained by GPU memory. This constraint is mitigated by expert offloading, which stores part of the model's weight in host memory and executes on demand. Though MoE architecture has computation sparsity, more GPUs are…
Excerpt provided by arXiv cs.AI via its public feed. Full reporting belongs to the original publisher.
Read the full story on arXiv cs.AI
Related coverage
The Decoder · 40m ago
Gemini 4 Carbon arrives just in time to feel like yesterdays news in the fastest model cycle yet
Ars Technica AI · 3h ago
DeepMind reveals Gemini Robotics 2.5 and open safety benchmark "ASIMOV-jailbreak"
Google DeepMind · 6h ago
Introducing AlphaProof Code: a breakthrough in verifying complex software at depth
OpenAI · 9h ago
Closing the AI guidance gap in education