vLLM Blog https://vllm.ai/blog Technical articles, release announcements, model guides, and community updates from the vLLM project. en-us Thu, 10 Sep 2026 21:45:48 GMT Following the Bottleneck: Optimizing MiniMax M3 on AMD Instinct MI355X https://vllm.ai/blog/2026-09-10-minimax-m3-mi355x https://vllm.ai/blog/2026-09-10-minimax-m3-mi355x Thu, 10 Sep 2026 00:00:00 GMT A performance model for LLM serving: inspect local shapes, remove repeated work, verify data movement and dispatch, then follow the queue. hardware performance moe speculative-decoding disaggregation AMD and Embedded LLM Teams Tiered KV Cache Offloading in vLLM https://vllm.ai/blog/2026-09-10-tiered-kv-offloading https://vllm.ai/blog/2026-09-10-tiered-kv-offloading Thu, 10 Sep 2026 00:00:00 GMT A host-centric framework for scaling KV cache across host memory, filesystems, object stores, and remote peers — reducing recomputation and increasing serving capacity. Or Ozeri, Danny Harnik, Ronen Schaffer, Itay Etelis, Varun Sundar Rabindranath GLM 5.3 Optimizations, Part 1: Hybrid HiSparse Offloading in vLLM https://vllm.ai/blog/2026-09-08-glm53-part1-hybrid-sparse-offloading https://vllm.ai/blog/2026-09-08-glm53-part1-hybrid-sparse-offloading Tue, 08 Sep 2026 00:00:00 GMT vLLM integrates HiSparse as a pressure-driven memory tier that composes with the Hybrid Memory Allocator and KV offloading, letting GLM 5.3 requests keep decoding when their KV no longer fits in GPU memory, so concurrency stays high. glm kv-cache performance vLLM Team vLLM x AgentX: Optimizing for Real-World Agentic Serving https://vllm.ai/blog/2026-09-08-vllm-agentx https://vllm.ai/blog/2026-09-08-vllm-agentx Tue, 08 Sep 2026 00:00:00 GMT How vLLM optimizes KV cache management, parallelism, scheduling, and P/D disaggregation for agentic workloads, validated on SemiAnalysis AgentX with up to 130K tokens per GPU-second and a 14.6x-106x serving-cost advantage over Opus 5. agentic kv_cache parallelism large-scale-serving disaggregation performance vLLM Team and Inferact Serving LLMs on Tenstorrent Hardware: Inside the vLLM TT Plugin https://vllm.ai/blog/2026-09-07-vllm-tt-plugin https://vllm.ai/blog/2026-09-07-vllm-tt-plugin Mon, 07 Sep 2026 00:00:00 GMT Tenstorrent accelerators join vLLM as an out-of-tree platform plugin, driven by mesh-architecture choices: phase-based scheduling, single-process data parallelism on Galaxy, on-device sampling with host fallback, and async decode overlap. hardware ecosystem Tenstorrent Team MiniMax H3 on vLLM-Omni: From System-Wide Optimization to Real-Time Serving with FastVideo’s FastH3 https://vllm.ai/blog/2026-09-01-minimax-h3-production-serving https://vllm.ai/blog/2026-09-01-minimax-h3-production-serving Tue, 01 Sep 2026 00:00:00 GMT How vLLM-Omni optimizes and scales the complete MiniMax H3 stack, then integrates FastVideo’s four-step FastH3 for generation faster than playback. performance large-scale-serving multimodal vllm-omni fastvideo fasth3 vLLM-Omni Team Exploring Speculative Decoding in vLLM on AMD GPUs https://vllm.ai/blog/2026-08-23-speculative-decoding-amd-gpus https://vllm.ai/blog/2026-08-23-speculative-decoding-amd-gpus Sun, 23 Aug 2026 00:00:00 GMT A practical guide to speculative decoding in vLLM on AMD GPUs, covering draft-and-verify mechanics, MTP, EAGLE-3, DFlash, DSpark, configuration, tuning, and benchmark results. speculative-decoding amd AMD and Embedded LLM Large-Scale Sharded Weight Transfer with Ray Direct Transport (RDT) in vLLM https://vllm.ai/blog/2026-08-22-rdt-weight-transfer https://vllm.ai/blog/2026-08-22-rdt-weight-transfer Sat, 22 Aug 2026 00:00:00 GMT We implement a native sharded weight transfer engine in vLLM utilizing Ray Direct Transport (RDT), achieving weight transfer for the Kimi K2 model in BF16 on 48 8xH100 nodes in 7.53s reinforcement-learning performance Aaron Hao, Sumanth Hegde, Gal Meirom, Istvan Haller, Kourosh Hakhamaneshi, Gavin Parnaby, Moein Khazraee, Omri Kahalon IsoExec: Unified Execution to Eliminate Trainer-Inference Mismatch in SkyRL https://vllm.ai/blog/2026-08-21-isoexec https://vllm.ai/blog/2026-08-21-isoexec Fri, 21 Aug 2026 00:00:00 GMT IsoExec unifies numerical execution across SkyRL's vLLM and Megatron runtimes, reducing the average rollout-versus-training logprob difference below 1e-6 on Qwen3.5-35B-A3B with 25% overhead. reinforcement-learning performance Alexander Jiang and the SkyRL Team VeRL-Omni v0.2.0: Faster Diffusion RL and Stable Omni Training https://vllm.ai/blog/2026-08-20-verl-omni-v0-2-0 https://vllm.ai/blog/2026-08-20-verl-omni-v0-2-0 Thu, 20 Aug 2026 00:00:00 GMT A release focused on higher-throughput diffusion rollout, reusable omni adapters, and broader recipe coverage. multimodal rlhf ecosystem performance VeRL-Omni Team Distributed Layerwise Offload: Scaling Toward 200B+ DiT Models Efficiently in vLLM-Omni https://vllm.ai/blog/2026-08-17-distributed-layerwise-offload https://vllm.ai/blog/2026-08-17-distributed-layerwise-offload Mon, 17 Aug 2026 00:00:00 GMT Distributed Layerwise Offload shards and streams DiT weights across devices, serving a measured 124 GB Cosmos3 model on 64 GB HBM and estimating a path toward 200B+ models. performance distributed vllm-omni cosmos3 vLLM-Omni Diffusion Team Adaptive Verification in vLLM: DSpark confidence-scheduled verification https://vllm.ai/blog/2026-08-14-dspark-adaptive-verification https://vllm.ai/blog/2026-08-14-dspark-adaptive-verification Fri, 14 Aug 2026 00:00:00 GMT Sizing the DSpark draft-verification budget from per-request confidence instead of verifying every drafted token, so one configuration holds the throughput/latency frontier from batch size 1 to 256. performance speculative-decoding vLLM Team Day 0 Support for Qwen3.8-2.4T-A95B on vLLM https://vllm.ai/blog/2026-08-12-qwen3.8 https://vllm.ai/blog/2026-08-12-qwen3.8 Wed, 12 Aug 2026 00:00:00 GMT Day-0 vLLM support for Qwen3.8-2.4T-A95B: a 2.4-trillion-parameter hybrid MoE model served out of the box, with FP8/BF16 checkpoints plus NVFP4 and MXFP4 quantized weights, and co-developed kernels on NVIDIA and AMD hardware. model-support quantization moe performance hardware vLLM Team and Inferact Announcing Day-0 Support for NVIDIA Nemotron 3.5 Lightning on vLLM https://vllm.ai/blog/2026-08-10-nemotron-3-5-lightning-vllm https://vllm.ai/blog/2026-08-10-nemotron-3-5-lightning-vllm Mon, 10 Aug 2026 00:00:00 GMT How vLLM serves NVIDIA Nemotron 3.5 Lightning with OpenAI-compatible APIs, speculative decoding, and BF16/NVFP4 checkpoints across NVIDIA GPUs and edge systems. model-support NVIDIA Nemotron Team and vLLM Team Efficient Decode Context Parallelism with vLLM for Long Context Workloads https://vllm.ai/blog/2026-08-07-decode-context-parallelism https://vllm.ai/blog/2026-08-07-decode-context-parallelism Fri, 07 Aug 2026 00:00:00 GMT Decode Context Parallelism (DCP) in vLLM shards KV cache across GPUs by sequence dimension, enabling 3× higher throughput on long-context agentic workloads compared to standard tensor parallelism. performance attention parallelism Seonghee Lee, Sungsoo Ha, Omri Almog (NVIDIA), Lucas Wilkinson (Red Hat AI) vLLM Reaches 25K Total TPS/GPU on Qwen3.5 https://vllm.ai/blog/2026-08-06-qwen35-25k-tps https://vllm.ai/blog/2026-08-06-qwen35-25k-tps Thu, 06 Aug 2026 00:00:00 GMT How vLLM reaches 25K total TPS/GPU on Qwen3.5-397B-A17B-NVFP4 with GB200 NVL72 disaggregated serving, Blackwell GDN kernels, HMA cache transfer, async scheduling fixes, and srt-slurm recipes. performance qwen3.5 disaggregation vLLM Team Optimizing vLLM on Arm CPUs https://vllm.ai/blog/2026-07-29-optimizing-vllm-on-arm-cpus https://vllm.ai/blog/2026-07-29-optimizing-vllm-on-arm-cpus Wed, 29 Jul 2026 00:00:00 GMT An overview of Arm CPU enablement and inference performance optimizations in vLLM. hardware performance Arm Team Parallel All the Way Down: Beyond Single-Token Generation with Speculative Decoding https://vllm.ai/blog/2026-07-28-speculators-parallel-drafting https://vllm.ai/blog/2026-07-28-speculators-parallel-drafting Tue, 28 Jul 2026 00:00:00 GMT Speculators and vLLM now support P-EAGLE, DFlash, and DSpark — three parallel drafting algorithms that move beyond sequential token generation to deliver faster, simpler, and more scalable speculative decoding for LLM serving. speculators speculative_decoding peagle dflash dspark Alexandre Marques, Megan Flynn, Helen Zhao, Krishna Teja Chitty Venkata, Chibueze Ukachi (Red Hat AI) Kimi K3 Is Here: Efficient Day-0 Support on vLLM https://vllm.ai/blog/2026-07-27-k3 https://vllm.ai/blog/2026-07-27-k3 Mon, 27 Jul 2026 00:00:00 GMT vLLM delivers day-0 Kimi K3 serving with hybrid KDA prefix caching, DSpark speculative decoding, production-scale disaggregation, and optimized kernels across NVIDIA and AMD GPUs. models performance prefix caching multimodal vLLM Team and Inferact From Day 0 to Production SLAs: Serving GLM-5.2 on 24 NVIDIA B300 GPUs with vLLM https://vllm.ai/blog/2026-07-23-glm-5.2-nvfp4-b300-pd https://vllm.ai/blog/2026-07-23-glm-5.2-nvfp4-b300-pd Thu, 23 Jul 2026 00:00:00 GMT How we took GLM-5.2-NVFP4 from 40 ms to 17 ms mean TPOT on 24 B300 GPUs with vLLM: P/D disaggregation, MTP speculative decoding, Model Runner V2, and the SLA-first trade-offs behind the final configuration. disaggregation performance speculative-decoding moe large-scale-serving DaoCloud Team Announcing vLLM AFD Plugin: Disaggregating Attention and FFN for Flexible MoE Serving https://vllm.ai/blog/2026-07-23-vllm-afd-plugin https://vllm.ai/blog/2026-07-23-vllm-afd-plugin Thu, 23 Jul 2026 00:00:00 GMT What vLLM AFD Plugin adds to the vLLM ecosystem: Attention–FFN disaggregation for MoE serving, GPU and Ascend NPU backends, connector-based execution, and graph and ubatching support. inference moe ecosystem AFD Plugin Contributors A Preview of Production-Scale Kimi K3 Support on vLLM https://vllm.ai/blog/2026-07-22-kimi-k3-preview https://vllm.ai/blog/2026-07-22-kimi-k3-preview Wed, 22 Jul 2026 00:00:00 GMT A preview of production-scale Kimi K3 support in vLLM, including KDA-aware prefix caching, fused kernels, optimized MXFP4 MoE, multimodal integration, and initial NVIDIA and AMD paths. models performance prefix caching multimodal vLLM Team Beyond a Single Model: Building Mixture-of-Models Systems with vLLM Semantic Router https://vllm.ai/blog/2026-07-21-vllm-sr-new-chapter-mom https://vllm.ai/blog/2026-07-21-vllm-sr-new-chapter-mom Tue, 21 Jul 2026 00:00:00 GMT vLLM Semantic Router is expanding from intelligent routing into a system for building, evaluating, and running Mixture-of-Models. ecosystem mixture-of-models semantic-router vLLM Semantic Router Team Keeping vLLM Production Quality: A Look Inside CI, Benchmarking, and the Release Process https://vllm.ai/blog/2026-07-16-keeping-vllm-production-quality https://vllm.ai/blog/2026-07-16-keeping-vllm-production-quality Thu, 16 Jul 2026 00:00:00 GMT How vLLM maintains production quality with extensive CI across diverse accelerators, nightly performance benchmark and accuracy evaluation, and a two-week release process. ci performance evaluation release Kevin Luu (Inferact) TML Inkling on vLLM: Day-0 Support with Optimized Performance https://vllm.ai/blog/2026-07-15-inkling https://vllm.ai/blog/2026-07-15-inkling Wed, 15 Jul 2026 00:00:00 GMT vLLM brings day-0 support to TML Inkling, a 1T-parameter multimodal model, with MTP, long-context serving, parallelism, and up to 380 tokens per second per user on NVIDIA GB200 GPUs. model-support performance multimodal speculative-decoding vLLM Team vLLM x TileRT: Specialized Decode for Latency-Critical Serving https://vllm.ai/blog/2026-07-14-vllm-tilert-pd https://vllm.ai/blog/2026-07-14-vllm-tilert-pd Tue, 14 Jul 2026 00:00:00 GMT vLLM prefill paired with TileRT decode through vLLM V1's connector interface: a specialized, latency-optimized decode engine that coexists with native vLLM decode behind one shared serving layer, with zero changes to vLLM. disaggregation performance ecosystem TileRT team EAGLE3 Speculative Decoding on AMD Instinct GPUs: Training and Serving with vLLM and AMD Quark https://vllm.ai/blog/2026-07-13-eagle-3-amd-instinct https://vllm.ai/blog/2026-07-13-eagle-3-amd-instinct Mon, 13 Jul 2026 00:00:00 GMT How AMD Quark trains, quantizes, and serves EAGLE3 speculative-decoding drafts with vLLM on AMD Instinct GPUs, delivering up to 2.00x throughput gains for Kimi-K2.5 and 1.79x for MiniMax-M2.5. performance hardware Larry Li, Chao Li, Haichen Zhang, Chun Fang, Andy Luo, Spandan Tiwari, and Ashish Sirasao vime + ROCm: End-to-End RL Post-Training on AMD Instinct™ GPUs https://vllm.ai/blog/2026-07-10-vime-rocm https://vllm.ai/blog/2026-07-10-vime-rocm Fri, 10 Jul 2026 00:00:00 GMT Announcing ROCm support for vime, now running end-to-end on AMD Instinct MI355X GPUs with prebuilt container. reinforcement-learning hardware ecosystem post-training AMD contributors & vime community vLLM × HPC-Ops: High-Performance Attention and MoE Backends from Tencent Hunyuan https://vllm.ai/blog/2026-07-06-vllm-hpc-ops https://vllm.ai/blog/2026-07-06-vllm-hpc-ops Mon, 06 Jul 2026 00:00:00 GMT How HPC-Ops integrates Hopper-optimized attention and FP8 MoE backends into vLLM for Tencent Hunyuan Hy3, improving mixed-length decode, MoE latency, TTFT, and TPOT on NVIDIA H20. performance attention moe hpc-ops Tencent Hunyuan AI Infra Team and vLLM Team Experience and Lessons Learned from Serving Multi-Stage Qwen3-Omni in vLLM-Omni https://vllm.ai/blog/2026-07-01-qwen3-omni-optimization https://vllm.ai/blog/2026-07-01-qwen3-omni-optimization Wed, 01 Jul 2026 00:00:00 GMT How vLLM-Omni serves and optimizes Qwen3-Omni with staged Thinker-Talker-Code2Wav execution, batching, CUDA Graphs, async chunk, async output, replicas, hot-path cleanup, and perf validation. performance multimodal vllm-omni vLLM-Omni Team and Ant Group SCT Team Micro-Agent: Beat Frontier Models with Collaboration inside Model API https://vllm.ai/blog/2026-06-29-micro-agent-frontier-models https://vllm.ai/blog/2026-06-29-micro-agent-frontier-models Mon, 29 Jun 2026 00:00:00 GMT How vLLM Semantic Router turns vllm-sr/auto into a bounded micro-agent runtime for Confidence, Ratings, ReMoM, Fusion, Workflows, and benchmark-shaped collaboration. ecosystem agentic-routing vLLM Semantic Router Team Engineering TTS Inference in vLLM-Omni https://vllm.ai/blog/2026-06-23-vllm-omni-tts https://vllm.ai/blog/2026-06-23-vllm-omni-tts Tue, 23 Jun 2026 00:00:00 GMT How vLLM-Omni supports and optimizes Qwen3-TTS, VoxCPM2, Higgs Audio V3, and Fish Speech S2 Pro with staged serving, batching, CUDA Graphs, and model-specific kernels. performance multimodal inference vLLM-Omni TTS Team Beyond One Model: Fusion in vLLM Semantic Router https://vllm.ai/blog/2026-06-16-vllm-sr-fusion-api https://vllm.ai/blog/2026-06-16-vllm-sr-fusion-api Tue, 16 Jun 2026 00:00:00 GMT How vLLM Semantic Router Fusion runs a panel of models, uses a judge to analyze agreement and gaps, and synthesizes one answer while preserving routing policy, traces, and OpenAI-compatible serving. ecosystem vLLM Semantic Router Team MiniMax M3 in vLLM: Day-0 Serving for 1M-Token Multimodal Reasoning https://vllm.ai/blog/2026-06-12-minimax-m3-vllm https://vllm.ai/blog/2026-06-12-minimax-m3-vllm Fri, 12 Jun 2026 00:00:00 GMT How vLLM serves MiniMax M3 with MiniMax Sparse Attention, multimodal and reasoning parsers, MXFP8 weights, and long-context deployment recipes. minimax day-0-support moe long-context vLLM Team DiffusionGemma: The First Diffusion LLM (dLLM) Natively Supported in vLLM https://vllm.ai/blog/2026-06-10-diffusion-gemma https://vllm.ai/blog/2026-06-10-diffusion-gemma Wed, 10 Jun 2026 00:00:00 GMT How vLLM supports DiffusionGemma, the first native diffusion language model in vLLM, using Model Runner V2 state hooks, iterative denoising, bidirectional attention, and reused speculative decoding paths. model ecosystem inference The vLLM Team and Google DeepMind Team Announcing vime: A Simple, Stable, and Efficient RL Framework for LLMs https://vllm.ai/blog/2026-06-09-announcing-vime https://vllm.ai/blog/2026-06-09-announcing-vime Tue, 09 Jun 2026 00:00:00 GMT vime connects slime's training stack with vLLM rollouts to provide a simple, stable, and efficient RL post-training pipeline. reinforcement-learning ecosystem post-training vime Contributors and the vLLM Team vLLM Semantic Router v0.3 Themis: From Signals to Stateful Production Routing https://vllm.ai/blog/2026-06-05-v0.3-vllm-sr-themis-release https://vllm.ai/blog/2026-06-05-v0.3-vllm-sr-themis-release Fri, 05 Jun 2026 00:00:00 GMT What vLLM Semantic Router v0.3 Themis adds for production routing: canonical config, inspectable signal-decision-policy flows, safer operations, CLI/dashboard/Kubernetes alignment, and replayable routing behavior. ecosystem vLLM Semantic Router Team Announcing Day-0 Support for NVIDIA Nemotron 3 Ultra on vLLM https://vllm.ai/blog/2026-06-04-nemotron-3-ultra-vllm https://vllm.ai/blog/2026-06-04-nemotron-3-ultra-vllm Thu, 04 Jun 2026 00:00:00 GMT How to serve NVIDIA Nemotron 3 Ultra with vLLM for long-running agentic reasoning, including BF16 and NVFP4 checkpoints, supported GPU configurations, OpenAI-compatible deployment, and NeMo RL integration. model-support NVIDIA Nemotron Team Fast & Efficient LLM Inference with vLLM: A New Course with DeepLearning.AI https://vllm.ai/blog/2026-06-03-deeplearning-ai-vllm-course https://vllm.ai/blog/2026-06-03-deeplearning-ai-vllm-course Wed, 03 Jun 2026 00:00:00 GMT What the DeepLearning.AI vLLM course teaches: optimizing, deploying, and benchmarking LLM inference with LLM Compressor quantization, GuideLLM, KV cache sizing, serving, and memory tradeoffs. community ecosystem learning Cedric Clyburn Session-Aware Agentic Routing: Continuity-Aware Model Selection for Long-Horizon LLM Agents https://vllm.ai/blog/2026-06-02-session-aware-agentic-routing https://vllm.ai/blog/2026-06-02-session-aware-agentic-routing Tue, 02 Jun 2026 00:00:00 GMT How Session-Aware Agentic Routing in vLLM Semantic Router preserves long-horizon agent continuity with session memory, safe model-switch boundaries, prefix-cache-aware switch pricing, and replayable traces. ecosystem performance agentic-routing Xunzhuo Liu, Bowei He, Huamin Chen, Haichen Zhang (AMD), Andy Luo (AMD), and the vLLM Semantic Router Team Accelerating vLLM-Omni Inference with AutoRound Quantization https://vllm.ai/blog/2026-06-02-vllm-omni-autoround https://vllm.ai/blog/2026-06-02-vllm-omni-autoround Tue, 02 Jun 2026 00:00:00 GMT How AutoRound integrates with vLLM-Omni to serve W4A16 quantized multimodal, diffusion, image, and video models with smaller checkpoints, preserved quality, Intel XPU acceleration, and NVIDIA GPU support. quantization multimodal vllm-omni hardware vLLM-Omni Community, Intel AutoRound Team vLLM on the DGX Spark: Architecture, Configuration, and Local Evaluation https://vllm.ai/blog/2026-06-01-vllm-dgx-spark https://vllm.ai/blog/2026-06-01-vllm-dgx-spark Mon, 01 Jun 2026 00:00:00 GMT How to run vLLM on NVIDIA DGX Spark and GB10 systems, including unified memory behavior, NVFP4 Nemotron-3-Super serving, Docker deployment, Prometheus metrics, and local evaluation results. dgx-spark nemotron hardware deployment computex Inferact Accelerating Laguna XS.2 Inference with vLLM, Speculators, and LLM Compressor https://vllm.ai/blog/2026-05-28-laguna-xs2-dflash-llm-compressor https://vllm.ai/blog/2026-05-28-laguna-xs2-dflash-llm-compressor Thu, 28 May 2026 00:00:00 GMT How Laguna XS.2 is served and optimized in vLLM using first-class model integration, a DFlash speculator trained with Speculators, and FP8, NVFP4, INT4, and INT8 checkpoints from LLM Compressor. quantization speculative-decoding speculators llm-compressor dflash Megan Flynn, Dipika Sikka, Alexandre Marques Native RL APIs in vLLM https://vllm.ai/blog/2026-05-28-native-rl-apis https://vllm.ai/blog/2026-05-28-native-rl-apis Thu, 28 May 2026 00:00:00 GMT How vLLM native RL APIs standardize weight syncing and asynchronous RL serving with NCCL and CUDA IPC transfer backends, pause mode, and fixes for fragile DPEP and disaggregated rollout deployments. reinforcement-learning async-rl Aaron Hao, Sumanth Hegde, Kyle Sayers, Kourosh Hakhamaneshi, and the vLLM team Speculators v0.5.0: DFlash Support and Online Training https://vllm.ai/blog/2026-05-28-speculators-v050 https://vllm.ai/blog/2026-05-28-speculators-v050 Thu, 28 May 2026 00:00:00 GMT What Speculators v0.5.0 adds for vLLM speculative decoding: DFlash block-diffusion draft models, unified online and offline training, native hidden-state extraction, and Gemma 4 latency results. speculative-decoding ecosystem Fynn Schmitt-Ulms, Helen Zhao, Rahul Tuli and Dipika Sikka (Red Hat AI Model Optimization Team) From Text to Multimodal Routing: Hardening Vision Signals in vLLM Semantic Router https://vllm.ai/blog/2026-05-28-vllm-sr-vision-encoder-hardening https://vllm.ai/blog/2026-05-28-vllm-sr-vision-encoder-hardening Thu, 28 May 2026 00:00:00 GMT How vLLM Semantic Router hardens multimodal routing by turning visual evidence into trustworthy signals, debugging a Rust/Candle vision-encoder parity issue, and validating image signal correctness for production policy. ecosystem performance David Shrader, Huamin Chen, Xunzhuo Liu, Bowei He, and the vLLM Semantic Router Team EAGLE 3.1: Advancing Speculative Decoding Through Collaboration Between the EAGLE Team, vLLM, and TorchSpec https://vllm.ai/blog/2026-05-26-eagle-3-1 https://vllm.ai/blog/2026-05-26-eagle-3-1 Tue, 26 May 2026 00:00:00 GMT How EAGLE 3.1 improves speculative decoding robustness in vLLM with FC normalization, post-norm hidden-state feedback, TorchSpec training support, and config-driven compatibility with EAGLE 3 checkpoints. speculative-decoding performance EAGLE Team, vLLM Team, and TorchSpec Team vLLM x Novita AI: PegaFlow for Production-Grade External KV Cache https://vllm.ai/blog/2026-05-18-pegaflow https://vllm.ai/blog/2026-05-18-pegaflow Mon, 18 May 2026 00:00:00 GMT How PegaFlow integrates with vLLM as an external KV cache service, using a Rust daemon, CUDA IPC, RDMA, SSD caching, and the external KV connector to improve startup, sharing, throughput, and cache lifecycle. kv_cache disaggregation performance production-serving Novita AI and the vLLM Team Elastic Expert Parallelism in vLLM https://vllm.ai/blog/2026-05-14-elastic-expert-parallelism https://vllm.ai/blog/2026-05-14-elastic-expert-parallelism Thu, 14 May 2026 00:00:00 GMT How Elastic Expert Parallelism lets vLLM scale Mixture-of-Experts serving up or down at runtime by changing data-parallel workers, redistributing experts, and coordinating live topology changes without server restarts. large-scale-serving elastic-ep expert-parallelism moe fault-tolerance Itay Alroy (NVIDIA), Yongji Wu (Sky Computing), Rui Qiao (Anyscale), Tyler Michael Smith (Red Hat), Moein Khazraee (NVIDIA), Omri Kahalon (NVIDIA), Tzu-Ling Kan (NVIDIA), Ron Tourgeman (NVIDIA) Announcing VeRL-Omni: Easy, Fast, and Stable RL Training for Diffusion and Omni-Modality Models https://vllm.ai/blog/2026-05-14-verl-omni https://vllm.ai/blog/2026-05-14-verl-omni Thu, 14 May 2026 00:00:00 GMT How VeRL-Omni extends verl with vLLM-Omni for reinforcement learning post-training of diffusion and multimodal generative models, including efficient rollouts, reward inference, trainers, hardware support, and recipes. multimodal rlhf ecosystem VeRL-Omni Team