#disaggregation

6 posts

From Day 0 to Production SLAs: Serving GLM-5.2 on 24 NVIDIA B300 GPUs with vLLM

Jul 23, 2026·18 min read

How we took GLM-5.2-NVFP4 from 40 ms to 17 ms mean TPOT on 24 B300 GPUs with vLLM: P/D disaggregation, MTP speculative decoding, Model Runner V2, and the SLA-first trade-offs behind the final configuration.

vLLM x TileRT: Specialized Decode for Latency-Critical Serving

Jul 14, 2026·7 min read

vLLM prefill paired with TileRT decode through vLLM V1's connector interface: a specialized, latency-optimized decode engine that coexists with native vLLM decode behind one shared serving layer, with zero changes to vLLM.

vLLM x Novita AI: PegaFlow for Production-Grade External KV Cache

May 18, 2026·13 min read

How PegaFlow integrates with vLLM as an external KV cache service, using a Rust daemon, CUDA IPC, RDMA, SSD caching, and the external KV connector to improve startup, sharing, throughput, and cache lifecycle.

Serving Agentic Workloads at Scale with vLLM x Mooncake

May 6, 2026·10 min read

How vLLM integrates Mooncake Store as a distributed KV cache for agentic workloads, reusing shared prefixes across turns and instances to improve throughput, TTFT, end-to-end latency, and multi-GPU scaling.

Disaggregated Serving for Hybrid SSM Models in vLLM

Apr 21, 2026·15 min read

How vLLM extends NIXL prefill/decode disaggregation to hybrid SSM-attention models with dual descriptor views, physical-logical block bridging, and Mamba conv-state transfer support.

Next-Level Inference: Why Your Single-Node vLLM Setup Needs Prefill-Decode Disaggregation

Apr 7, 2026·22 min read

How single-node prefill/decode disaggregation in vLLM uses AMD MORI-IO on an 8-GPU MI300X node to separate prefill and decode, transfer KV cache efficiently, stabilize ITL, and improve goodput.