vLLM Blog https://vllm.ai/blog Technical articles, release announcements, model guides, and community updates from the vLLM project. en-us Wed, 10 Jun 2026 08:46:07 GMT Announcing vime: A Simple, Stable, and Efficient RL Framework for LLMs https://vllm.ai/blog/2026-06-09-announcing-vime https://vllm.ai/blog/2026-06-09-announcing-vime Tue, 09 Jun 2026 00:00:00 GMT vime connects slime's training stack with vLLM rollouts to provide a simple, stable, and efficient RL post-training pipeline. reinforcement-learning ecosystem post-training vime Contributors and the vLLM Team vLLM Semantic Router v0.3 Themis: From Signals to Stateful Production Routing https://vllm.ai/blog/2026-06-05-v0.3-vllm-sr-themis-release https://vllm.ai/blog/2026-06-05-v0.3-vllm-sr-themis-release Fri, 05 Jun 2026 00:00:00 GMT vLLM Semantic Router v0.3, codename Themis, is where semantic routing becomes stateful, observable, and production-ready for real AI traffic. ecosystem vLLM Semantic Router Team Announcing Day-0 Support for NVIDIA Nemotron 3 Ultra on vLLM https://vllm.ai/blog/2026-06-04-nemotron-3-ultra-vllm https://vllm.ai/blog/2026-06-04-nemotron-3-ultra-vllm Thu, 04 Jun 2026 00:00:00 GMT We are excited to announce Day-0 Support for the newly released NVIDIA Nemotron 3 Ultra on vLLM. model-support NVIDIA Nemotron Team Fast & Efficient LLM Inference with vLLM: A New Course with DeepLearning.AI https://vllm.ai/blog/2026-06-03-deeplearning-ai-vllm-course https://vllm.ai/blog/2026-06-03-deeplearning-ai-vllm-course Wed, 03 Jun 2026 00:00:00 GMT We're excited to announce, with Red Hat and Andrew Ng's DeepLearning.AI, a hands-on course that walks through LLM fundamentals and the full optimize, deploy, and benchmark AI deployment lifecycle... community ecosystem learning Cedric Clyburn Session-Aware Agentic Routing: Continuity-Aware Model Selection for Long-Horizon LLM Agents https://vllm.ai/blog/2026-06-02-session-aware-agentic-routing https://vllm.ai/blog/2026-06-02-session-aware-agentic-routing Tue, 02 Jun 2026 00:00:00 GMT Long-horizon LLM agents create a routing problem that single-turn prompt routers were not designed to solve. A router still needs to know which model is best for the current request, but it also... ecosystem performance agentic-routing Xunzhuo Liu, Bowei He, Huamin Chen, Haichen Zhang (AMD), Andy Luo (AMD), and the vLLM Semantic Router Team Accelerating vLLM-Omni Inference with AutoRound Quantization https://vllm.ai/blog/2026-06-02-vllm-omni-autoround https://vllm.ai/blog/2026-06-02-vllm-omni-autoround Tue, 02 Jun 2026 00:00:00 GMT We are excited to announce that AutoRound — Intel's state-of-the-art post-training quantization (PTQ) algorithm — is now fully integrated into vLLM-Omni, enabling a streamlined quantize-once,... quantization multimodal vllm-omni hardware vLLM-Omni Community, Intel AutoRound Team vLLM on the DGX Spark: Architecture, Configuration, and Local Evaluation https://vllm.ai/blog/2026-06-01-vllm-dgx-spark https://vllm.ai/blog/2026-06-01-vllm-dgx-spark Mon, 01 Jun 2026 00:00:00 GMT A technical deep dive on running vLLM on NVIDIA DGX Spark and GB10 systems, covering sm_121 architecture, unified memory behavior, NVFP4 model serving, Nemotron-3-Super configuration, Docker deployment, Prometheus metrics, and local evaluation results. dgx-spark nemotron hardware deployment computex Inferact Accelerating Laguna XS.2 Inference with vLLM, Speculators, and LLM Compressor https://vllm.ai/blog/2026-05-28-laguna-xs2-dflash-llm-compressor https://vllm.ai/blog/2026-05-28-laguna-xs2-dflash-llm-compressor Thu, 28 May 2026 00:00:00 GMT As organizations increasingly adopt AI-powered development tools, the need for high-performance agentic models that deliver both accuracy and operational efficiency has become critical. Laguna... quantization speculative-decoding speculators llm-compressor dflash Megan Flynn, Dipika Sikka, Alexandre Marques Native RL APIs in vLLM https://vllm.ai/blog/2026-05-28-native-rl-apis https://vllm.ai/blog/2026-05-28-native-rl-apis Thu, 28 May 2026 00:00:00 GMT As post-training workloads continue to scale, we've seen widespread adoption of vLLM as the inference engine of choice. However, two issues repeatedly arise: reinforcement-learning async-rl Aaron Hao, Sumanth Hegde, Kyle Sayers, Kourosh Hakhamaneshi, and the vLLM team Speculators v0.5.0: DFlash Support and Online Training https://vllm.ai/blog/2026-05-28-speculators-v050 https://vllm.ai/blog/2026-05-28-speculators-v050 Thu, 28 May 2026 00:00:00 GMT The v0.5.0 release brings significant architectural improvements to speculative decoding model training, introducing DFlash algorithm support, fully unified online training capabilities, and a... speculative-decoding ecosystem Fynn Schmitt-Ulms, Helen Zhao, Rahul Tuli and Dipika Sikka (Red Hat AI Model Optimization Team) From Text to Multimodal Routing: Hardening Vision Signals in vLLM Semantic Router https://vllm.ai/blog/2026-05-28-vllm-sr-vision-encoder-hardening https://vllm.ai/blog/2026-05-28-vllm-sr-vision-encoder-hardening Thu, 28 May 2026 00:00:00 GMT Most routing systems start with a prompt and choose a model endpoint. vLLM Semantic Router (VSR) makes a different bet: before a request reaches the serving model, the system should extract... ecosystem performance David Shrader, Huamin Chen, Xunzhuo Liu, Bowei He, and the vLLM Semantic Router Team EAGLE 3.1: Advancing Speculative Decoding Through Collaboration Between the EAGLE Team, vLLM, and TorchSpec https://vllm.ai/blog/2026-05-26-eagle-3-1 https://vllm.ai/blog/2026-05-26-eagle-3-1 Tue, 26 May 2026 00:00:00 GMT The EAGLE series — including EAGLE 1, EAGLE 2, and EAGLE 3 — has become one of the most widely adopted and practically deployed families of speculative decoding algorithms across both research and... speculative-decoding performance EAGLE Team, vLLM Team, and TorchSpec Team vLLM x Novita AI: PegaFlow for Production-Grade External KV Cache https://vllm.ai/blog/2026-05-18-pegaflow https://vllm.ai/blog/2026-05-18-pegaflow Mon, 18 May 2026 00:00:00 GMT TL;DR: In collaboration with Novita AI, PegaFlow integrates with vLLM as an external KV cache service for LLM inference, implemented as a standalone Rust process and connected through the external... kv_cache disaggregation performance production-serving Novita AI and the vLLM Team Elastic Expert Parallelism in vLLM https://vllm.ai/blog/2026-05-14-elastic-expert-parallelism https://vllm.ai/blog/2026-05-14-elastic-expert-parallelism Thu, 14 May 2026 00:00:00 GMT Expert parallelism (EP) is a key technique for serving Mixture-of-Experts (MoE) models at high throughput. WideEP deployments (where EP spans many workers) maximize KV cache capacity, enabling... large-scale-serving elastic-ep expert-parallelism moe fault-tolerance Itay Alroy (NVIDIA), Yongji Wu (Sky Computing), Rui Qiao (Anyscale), Tyler Michael Smith (Red Hat), Moein Khazraee (NVIDIA), Omri Kahalon (NVIDIA), Tzu-Ling Kan (NVIDIA), Ron Tourgeman (NVIDIA) Announcing VeRL-Omni: Easy, Fast, and Stable RL Training for Diffusion and Omni-Modality Models https://vllm.ai/blog/2026-05-14-verl-omni https://vllm.ai/blog/2026-05-14-verl-omni Thu, 14 May 2026 00:00:00 GMT We are excited to announce the pre-release of VeRL-Omni, a general reinforcement learning (RL) post-training framework focused on multimodal generative models, built on top of verl and vllm-omni. multimodal rlhf ecosystem VeRL-Omni Team A First Comprehensive Study of TurboQuant: Accuracy and Performance https://vllm.ai/blog/2026-05-11-turboquant https://vllm.ai/blog/2026-05-11-turboquant Mon, 11 May 2026 00:00:00 GMT TurboQuant, a method for KV-cache quantization, recently gained significant traction in the community due to the large advertised savings in GPU memory from very low bit-width quantization of a... quantization kv_cache turboquant Eldar Kurtić, Michael Goin, Alexandre Marques (Red Hat AI) vLLM Tops the Artificial Analysis Leaderboard https://vllm.ai/blog/2026-05-11-vllm-tops-artificial-analysis https://vllm.ai/blog/2026-05-11-vllm-tops-artificial-analysis Mon, 11 May 2026 00:00:00 GMT How vLLM built the leading deployments of DeepSeek V3.2, MiniMax-M2.5, and Qwen 3.5 397B. performance benchmarking kernel-fusion speculative-decoding vLLM Team Serving Agentic Workloads at Scale with vLLM x Mooncake https://vllm.ai/blog/2026-05-06-mooncake-store https://vllm.ai/blog/2026-05-06-mooncake-store Wed, 06 May 2026 00:00:00 GMT TL;DR: Agentic workloads generate massive shared prefixes that are often recomputed across turns. By integrating Mooncake's distributed KV cache store into vLLM, we achieve 3.8x higher throughput,... agentic kv_cache large-scale-serving disaggregation Yifan Qiao, Trong Dao Le, Ao Shen, Zhewen Li, Bowen Wang Run Highly Efficient Multimodal Agentic AI with NVIDIA Nemotron 3 Nano Omni Using vLLM https://vllm.ai/blog/2026-04-28-nemotron-omni https://vllm.ai/blog/2026-04-28-nemotron-omni Tue, 28 Apr 2026 00:00:00 GMT We are excited to support the newly released NVIDIA Nemotron 3 Nano Omni model on vLLM. model-support NVIDIA Nemotron Team DeepSeek V4 in vLLM: Efficient Long-context Attention https://vllm.ai/blog/2026-04-24-deepseek-v4 https://vllm.ai/blog/2026-04-24-deepseek-v4 Fri, 24 Apr 2026 00:00:00 GMT A first-principles walkthrough of DeepSeek V4's long-context attention, and how we implemented it in vLLM. model-support vLLM Team The State of FP8 KV-Cache and Attention Quantization in vLLM https://vllm.ai/blog/2026-04-22-fp8-kvcache https://vllm.ai/blog/2026-04-22-fp8-kvcache Wed, 22 Apr 2026 00:00:00 GMT Long-context LLM serving is increasingly memory-bound: for standard full-attention decoders, the KV cache often dominates GPU memory at 128k+ contexts, and each decode step must read a large... quantization performance kv_cache fp8 Jonas Kübler* (AWS), Eldar Kurtić* (Red Hat AI), Lucas Wilkinson (Red Hat AI), Matthew Bonanni (Red Hat AI), Michael Goin (Red Hat AI), Alexandre Marques (Red Hat AI), Kailash Budhathoki (AWS) (* Equal Contribution) Disaggregated Serving for Hybrid SSM Models in vLLM https://vllm.ai/blog/2026-04-21-hybrid-ssm-disagg https://vllm.ai/blog/2026-04-21-hybrid-ssm-disagg Tue, 21 Apr 2026 00:00:00 GMT Hybrid architectures that interleave Mamba-style SSM layers with standard full-attention (FA) layers — such as NVIDIA Nemotron-H — are gaining traction as a way to combine the linear-time... disaggregation mamba Nicolò Lucchesi, Zhanqiu Hu (Red Hat), and the vLLM team vLLM Korea Meetup 2026 Wrap-Up https://vllm.ai/blog/2026-04-14-vllm-korea-meetup-2026 https://vllm.ai/blog/2026-04-14-vllm-korea-meetup-2026 Tue, 14 Apr 2026 00:00:00 GMT Hosted by the vLLM KR Community, with support from Rebellions, SqueezeBits, Red Hat APAC, and PyTorch Korea, the vLLM Korea Meetup 2026 was held in Seoul on April 2nd. community vLLM Team Next-Level Inference: Why Your Single-Node vLLM Setup Needs Prefill-Decode Disaggregation https://vllm.ai/blog/2026-04-07-moriio-kv-connector https://vllm.ai/blog/2026-04-07-moriio-kv-connector Tue, 07 Apr 2026 00:00:00 GMT TL;DR: Prefill and decode fight over the same GPUs, causing ITL spikes under load. We show how to disaggregate them on a single 8-GPU MI300X node using AMD's MORI-IO connector — achieving 2.5x... disaggregation AMD and Embedded LLM Announcing Gemma 4 on vLLM: Byte for byte, the most capable open models https://vllm.ai/blog/2026-04-02-gemma4 https://vllm.ai/blog/2026-04-02-gemma4 Thu, 02 Apr 2026 00:00:00 GMT With the debut of Gemma 4, vLLM introduces immediate support for Google's most sophisticated open model lineup, spanning multiple hardware backends, with first-ever Day 0 support on Google TPUs,... model-support Google Team Extracting hidden states from vLLM https://vllm.ai/blog/2026-03-30-extract-hidden-states https://vllm.ai/blog/2026-03-30-extract-hidden-states Mon, 30 Mar 2026 00:00:00 GMT PR #33736 (included in vllm>=v0.18.0) introduced a new hidden states extraction system to vLLM. This blog post explores the motivation, design, usage, and future direction of this feature, and its... speculative-decoding Fynn Schmitt-Ulms Model Runner V2: A Modular and Faster Core for vLLM https://vllm.ai/blog/2026-03-24-mrv2 https://vllm.ai/blog/2026-03-24-mrv2 Tue, 24 Mar 2026 00:00:00 GMT We are excited to announce Model Runner V2 (MRV2), a ground-up re-implementation of the vLLM model runner. MRV2 delivers a cleaner, more modular, and more efficient execution core—with no API... performance engineering vLLM Team P-EAGLE: Faster LLM inference with Parallel Speculative Decoding in vLLM https://vllm.ai/blog/2026-03-13-p-eagle https://vllm.ai/blog/2026-03-13-p-eagle Fri, 13 Mar 2026 00:00:00 GMT EAGLE is the state-of-the-art method for speculative decoding in large language model (LLM) inference, but its autoregressive drafting creates a hidden bottleneck: the more tokens that you... performance speculative-decoding Amazon and NVIDIA Team Run Highly Efficient and Accurate Multi-Agent AI with NVIDIA Nemotron 3 Super Using vLLM https://vllm.ai/blog/2026-03-11-nemotron-3-super https://vllm.ai/blog/2026-03-11-nemotron-3-super Wed, 11 Mar 2026 00:00:00 GMT We are excited to support the newly released NVIDIA Nemotron 3 Super model on vLLM. model-support NVIDIA Nemotron Team vLLM Semantic Router v0.2 Athena: ClawOS, Model Refresh, and the System Brain https://vllm.ai/blog/2026-03-10-v0.2-vllm-sr-athena-release https://vllm.ai/blog/2026-03-10-v0.2-vllm-sr-athena-release Tue, 10 Mar 2026 00:00:00 GMT Since v0.1 Iris, vLLM Semantic Router has made a large jump. In one release cycle, the project rebuilt its model stack, expanded routing into safety, semantic caching, memory, retrieval, and... ecosystem vLLM Semantic Router Team vLLM Triton Attention Backend Deep Dive https://vllm.ai/blog/2026-03-04-vllm-triton-backend-deep-dive https://vllm.ai/blog/2026-03-04-vllm-triton-backend-deep-dive Wed, 04 Mar 2026 00:00:00 GMT This article is adapted from a Red Hat hosted vLLM Office Hours session with Burkhard Ringlein from IBM Research, featuring a deep technical walkthrough of the vLLM Triton attention backend.... performance triton attention vLLM Team at IBM Research Beyond Porting: How vLLM Orchestrates High-Performance Inference on AMD ROCm https://vllm.ai/blog/2026-02-27-rocm-attention-backend https://vllm.ai/blog/2026-02-27-rocm-attention-backend Fri, 27 Feb 2026 00:00:00 GMT For a long time, enabling AMD support meant "porting"; i.e. just making code run. That era is over. performance hardware AMD and Embedded LLM Efficiently serve dozens of fine-tuned models with vLLM on Amazon SageMaker AI and Amazon Bedrock https://vllm.ai/blog/2026-02-26-multi-lora https://vllm.ai/blog/2026-02-26-multi-lora Thu, 26 Feb 2026 00:00:00 GMT Organizations and individuals running multiple custom AI models, especially recent Mixture of Experts (MoE) model families, can face the challenge of paying for idle GPU capacity when the... performance Danielle Maddix Robinson, Florian Saupe, George Novack, Haipeng Li, Mani Kumar Adari, Xiang Song, Yu Gong (AWS AI Team) DeepSeek-V3.2 on GB300: Performance Breakthrough https://vllm.ai/blog/2026-02-13-gb300-deepseek https://vllm.ai/blog/2026-02-13-gb300-deepseek Fri, 13 Feb 2026 00:00:00 GMT DeepSeek-V3.2 (NVFP4 + TP2)has been successfully and smoothly run on GB300 (SM103 - Blackwell Ultra). Leveraging FP4 quantization, it achieves a single-GPU throughput of 7360 TGS (tokens / GPU /... hardware quantization performance The DaoCloud and vLLM team Driving vLLM WideEP and Large-Scale Serving Toward Maturity on Blackwell (Part I) https://vllm.ai/blog/2026-02-03-dsr1-gb200-part1 https://vllm.ai/blog/2026-02-03-dsr1-gb200-part1 Tue, 03 Feb 2026 00:00:00 GMT Building on our previous work achieving 2.2k tok/s/H200 decode throughput with wide-EP, the vLLM team has continued performance optimization efforts targeting NVIDIA's GB200 platform. This blog... large-scale-serving performance hardware Meta and NVIDIA Team GPT-OSS Performance Optimizations on NVIDIA Blackwell: Pushing the Pareto Frontier https://vllm.ai/blog/2026-02-01-gpt-oss-optimizations https://vllm.ai/blog/2026-02-01-gpt-oss-optimizations Sun, 01 Feb 2026 00:00:00 GMT TL;DR: In collaboration with the open-source community, vLLM + NVIDIA has achieved significant performance milestones on the gpt-oss-120b model running on NVIDIA's Blackwell GPUs. Through deep... performance hardware The vLLM and NVIDIA team Streaming Requests & Realtime API in vLLM https://vllm.ai/blog/2026-01-31-streaming-realtime https://vllm.ai/blog/2026-01-31-streaming-realtime Sat, 31 Jan 2026 00:00:00 GMT Large language model inference has traditionally operated on a simple premise: the user submits a complete prompt (request), the model processes it, and returns a response (either streaming or at... multimodal Meta, Mistral AI as well as the vLLM team Building Mixture-of-Models on AMD GPUs with vLLM-SR https://vllm.ai/blog/2026-01-23-mom-on-amd-gpu https://vllm.ai/blog/2026-01-23-mom-on-amd-gpu Fri, 23 Jan 2026 00:00:00 GMT We are working on building the System Level Intelligence for Mixture-of-Models (MoM), bringing Collective Intelligence into LLM systems. hardware ecosystem The AMD and vLLM Semantic Router Team Inside vLLM’s New KV Offloading Connector: Smarter Memory Transfer for Maximizing Inference Throughput https://vllm.ai/blog/2026-01-08-kv-offloading-connector https://vllm.ai/blog/2026-01-08-kv-offloading-connector Thu, 08 Jan 2026 00:00:00 GMT In this post, we will describe the new KV cache offloading feature that was introduced in vLLM 0.11.0. We will focus on offloading to CPU memory (DRAM) and its benefits to improving overall... performance Or Ozeri, Danny Harnik (vLLM Team at IBM Research) vLLM Semantic Router v0.1 Iris: The First Major Release https://vllm.ai/blog/2026-01-05-vllm-sr-iris https://vllm.ai/blog/2026-01-05-vllm-sr-iris Mon, 05 Jan 2026 00:00:00 GMT vLLM Semantic Router is the System Level Intelligence for Mixture-of-Models (MoM), bringing Collective Intelligence into LLM systems. It lives between users and models, capturing signals from... ecosystem vLLM Semantic Router Team Introducing vLLM Playground: A Modern Web Interface for Managing and Interacting with vLLM Servers https://vllm.ai/blog/2026-01-02-introducing-vllm-playground https://vllm.ai/blog/2026-01-02-introducing-vllm-playground Fri, 02 Jan 2026 00:00:00 GMT As a passionate vLLM community member who wants to see vLLM thrive and reach even more developers, I'm excited to announce vLLM Playground – a modern, feature-rich web interface for managing and... frontend ecosystem micytao Announcing vllm.ai Website and Some Community Updates https://vllm.ai/blog/2025-12-27-vllm-ai-website https://vllm.ai/blog/2025-12-27-vllm-ai-website Sat, 27 Dec 2025 00:00:00 GMT For a long time, vllm.ai simply redirected to the vLLM GitHub page. Thanks to our community, we now have a brand-new vllm.ai website, drawing inspiration from the PyTorch website. community vLLM Team vLLM-Omni Diffusion Cache Acceleration https://vllm.ai/blog/2025-12-19-vllm-omni-diffusion-cache-acceleration https://vllm.ai/blog/2025-12-19-vllm-omni-diffusion-cache-acceleration Fri, 19 Dec 2025 00:00:00 GMT We are thrilled to announce a major performance update for vLLM-Omni. multimodal performance ecosystem vLLM-Omni Team vLLM Large Scale Serving: DeepSeek @ 2.2k tok/s/H200 with Wide-EP https://vllm.ai/blog/2025-12-17-large-scale-serving https://vllm.ai/blog/2025-12-17-large-scale-serving Wed, 17 Dec 2025 00:00:00 GMT In v0.11.0, the last code from vLLM V0 engine was removed, marking the complete migration to the improved V1 engine architecture. This achievement would not have been possible without vLLM’s... large-scale-serving performance vLLM Team AMD × vLLM Semantic Router: Building the System Intelligence Together https://vllm.ai/blog/2025-12-16-vllm-sr-amd https://vllm.ai/blog/2025-12-16-vllm-sr-amd Tue, 16 Dec 2025 00:00:00 GMT Over the past several months, AMD and the vLLM SR Team have been collaborating to bring vLLM Semantic Router (VSR) to AMD GPUs—not just as a performance optimization, but as a fundamental shift in... hardware ecosystem The AMD and vLLM Semantic Router Team Run Highly Efficient and Accurate AI Agents with NVIDIA Nemotron 3 Nano on vLLM https://vllm.ai/blog/2025-12-15-run-nvidia-nemotron-3-nano https://vllm.ai/blog/2025-12-15-run-nvidia-nemotron-3-nano Mon, 15 Dec 2025 00:00:00 GMT Jan 28th Update: NVIDIA just released their Nemotron 3 Nano model in NVFP4 precision. This model is supported by vLLM out of the box and it uses a new method called Quantization-Aware Distillation... model-support NVIDIA Nemotron Team Encoder Disaggregation for Scalable Multimodal Model Serving https://vllm.ai/blog/2025-12-15-vllm-epd https://vllm.ai/blog/2025-12-15-vllm-epd Mon, 15 Dec 2025 00:00:00 GMT Modern Large Multimodal Models (LMMs) introduce a unique serving-time bottleneck: before any text generation can begin, all images must be processed by a visual encoder (e.g., ViT). This encoder... multimodal large-scale-serving Multimodality Workstream @ vLLM Token-Level Truth: Real-Time Hallucination Detection for Production LLMs https://vllm.ai/blog/2025-12-14-halugate https://vllm.ai/blog/2025-12-14-halugate Sun, 14 Dec 2025 00:00:00 GMT Your LLM just called a tool, received accurate data, and still got the answer wrong. Welcome to the world of extrinsic hallucination—where models confidently ignore the ground truth sitting right... ecosystem vLLM Semantic Router Team Diving into speculative decoding training support for vLLM with Speculators v0.3.0 https://vllm.ai/blog/2025-12-13-speculators-v030 https://vllm.ai/blog/2025-12-13-speculators-v030 Sat, 13 Dec 2025 00:00:00 GMT - Speculative decoding serves as an optimization to improve inference performance; however, training a unique draft model for each LLM can be difficult and time-consuming, while production-ready... speculative-decoding ecosystem Fynn Schmitt-Ulms, Helen Zhao, Rahul Tuli and Dipika Sikka (Red Hat AI Model Optimization Team) vLLM Router: A High-Performance and Prefill/Decode Aware Load Balancer for Large-scale Serving https://vllm.ai/blog/2025-12-13-vllm-router-release https://vllm.ai/blog/2025-12-13-vllm-router-release Sat, 13 Dec 2025 00:00:00 GMT Efficiently managing request distribution across a fleet of model replicas is a critical requirement for large-scale, production vLLM deployments. Standard load balancers often fall short as they... large-scale-serving vLLM Team