#moe

5 posts

From Day 0 to Production SLAs: Serving GLM-5.2 on 24 NVIDIA B300 GPUs with vLLM

Jul 23, 2026·18 min read

How we took GLM-5.2-NVFP4 from 40 ms to 17 ms mean TPOT on 24 B300 GPUs with vLLM: P/D disaggregation, MTP speculative decoding, Model Runner V2, and the SLA-first trade-offs behind the final configuration.

Announcing vLLM AFD Plugin: Disaggregating Attention and FFN for Flexible MoE Serving

Jul 23, 2026·9 min read

What vLLM AFD Plugin adds to the vLLM ecosystem: Attention–FFN disaggregation for MoE serving, GPU and Ascend NPU backends, connector-based execution, and graph and ubatching support.

vLLM × HPC-Ops: High-Performance Attention and MoE Backends from Tencent Hunyuan

Jul 6, 2026·12 min read

How HPC-Ops integrates Hopper-optimized attention and FP8 MoE backends into vLLM for Tencent Hunyuan Hy3, improving mixed-length decode, MoE latency, TTFT, and TPOT on NVIDIA H20.

MiniMax M3 in vLLM: Day-0 Serving for 1M-Token Multimodal Reasoning

Jun 12, 2026·21 min read

How vLLM serves MiniMax M3 with MiniMax Sparse Attention, multimodal and reasoning parsers, MXFP8 weights, and long-context deployment recipes.

Elastic Expert Parallelism in vLLM

May 14, 2026·11 min read

How Elastic Expert Parallelism lets vLLM scale Mixture-of-Experts serving up or down at runtime by changing data-parallel workers, redistributing experts, and coordinating live topology changes without server restarts.