All Posts
159 articles in chronological order.
2026(101)
- Day-0 Support for Flint on vLLM
Inferact and Recursive
- Announcing Humming: Fast, Flexible Kernels for Quantized Inference
Ant Group and the vLLM Community
- Introducing vLLM TorchTPU
Inferact and Google Teams
- Achieving near-zero-downtime scaling in Elastic Expert Parallelism
Itay Alroy (NVIDIA)
- Hyper Optimizing Kimi K3 in vLLM: Moving Fast and Responsibly
Benjamin Chislett (NVIDIA)
- Scaling Kimi K3 on AMD Instinct MI355X: Matching NVIDIA HGX B300 Throughput per GPU on AgentX
AMD and Embedded LLM
- Serving Reflection Beam on vLLM from Day 0
Inferact, Reflection, and Nvidia
- Day 0 support for Muse Spark Open on vLLM
vLLM Team
- Announcing Day 0 Support for NVIDIA Nemotron 3.5 Super VL on vLLM
NVIDIA Nemotron Team
- Adaptive Verification Without a Confidence Head
Giancarlo Delfin (Inferact)
- Push-Based KV Transfer to Improve TTFT and TPOT for Disaggregated Inference in vLLM
Sunita Nadampalli (AWS), Nicolò Lucchesi (Mistral)
- Finish RL Training Faster: Bitwise-Consistent, High-Performance, Agent-Driven
Yan Bai (NVIDIA DevTech), Ao Shen (Inferact)
- Improve TTFT for Multi-Turn Agentic Workloads with Bi-Directional KV Transfer in vLLM
Sunita Nadampalli (AWS)
- vLLM Support for NVIDIA Vera Rubin NVL72: 7.8x Throughput over GB200 NVL72
vLLM Team, Inferact, Red Hat, and NVIDIA
- DeepSeek-V4.1-Flash on vLLM: 5x Agentic Throughput Since Day 0
Inferact and the vLLM Team
- Taking vLLM Apart: A Practical Guide to Disaggregated Serving
Martin Hickey (IBM Research)
- Watermarking in vLLM
Raphaël Rialland (Mistral), Simon Veitner (Red Hat), and Tomas Ruiz (Red Hat)
- Reducing vLLM startup time with one uv flag
Nils Matteson
- Announcing vllm-metal: Concurrent Serving on Apple Silicon
Ranran Haoran Zhang, Lik Xun Yuan, Chao Ju Chen, Eric Curtin, Michael Goin
- PD Serving of Qwen3.8-2.4T
vLLM Team
- Scaling Multi-GPU Video Captioning with PyNvVideoCodec and vLLM
NVIDIA Computer Vision Team (NVCV)
- How we trained the fastest DSpark for Kimi-K3 using GB300 NVL72
Helen Zhao, Fynn Schmitt-Ulms, Yuchen Fama, Antonio J. Dominguez, and Kevin Li
- vLLM x Novita AI: Chord, Faster INT4 MoE for Kimi K2.x. Up to 1.3x on H200, 2.15x on Untuned B300
Novita AI and the vLLM Team
- vime × RL-Kernel × AMD: Bitwise Train–Rollout Consistency on ROCm
RL-Kernel Team, vime Team, and AMD Team
- Kimi K3 Performance Optimizations in vLLM: The Road to 2.8× Throughput
Wentao Ye (Red Hat), Canlin Guo (Huawei), Yongye Zhu (Inferact), Jiangyun Zhu (Inferact), Ziming Huang (Inferact), Wei Zhao (NVIDIA), Michael Goin (Red Hat), Jie Li (Inferact)
- Following the Bottleneck: Optimizing MiniMax M3 on AMD Instinct MI355X
AMD and Embedded LLM Teams
- Tiered KV Cache Offloading in vLLM
Or Ozeri, Danny Harnik, Ronen Schaffer, Itay Etelis, Varun Sundar Rabindranath
- GLM 5.3 Optimizations, Part 1: Hybrid HiSparse Offloading in vLLM
vLLM Team
- vLLM x AgentX: Optimizing for Real-World Agentic Serving
vLLM Team and Inferact
- Serving LLMs on Tenstorrent Hardware: Inside the vLLM TT Plugin
Tenstorrent Team
- MiniMax H3 on vLLM-Omni: From System-Wide Optimization to Real-Time Serving with FastVideo’s FastH3
vLLM-Omni Team
- Exploring Speculative Decoding in vLLM on AMD GPUs
AMD and Embedded LLM
- Large-Scale Sharded Weight Transfer with Ray Direct Transport (RDT) in vLLM
Aaron Hao, Sumanth Hegde, Gal Meirom, Istvan Haller, Kourosh Hakhamaneshi, Gavin Parnaby, Moein Khazraee, Omri Kahalon
- IsoExec: Unified Execution to Eliminate Trainer-Inference Mismatch in SkyRL
Alexander Jiang and the SkyRL Team
- VeRL-Omni v0.2.0: Faster Diffusion RL and Stable Omni Training
VeRL-Omni Team
- Distributed Layerwise Offload: Scaling Toward 200B+ DiT Models Efficiently in vLLM-Omni
vLLM-Omni Diffusion Team
- Adaptive Verification in vLLM: DSpark confidence-scheduled verification
vLLM Team
- Day 0 Support for Qwen3.8-2.4T-A95B on vLLM
vLLM Team and Inferact
- Announcing Day-0 Support for NVIDIA Nemotron 3.5 Lightning on vLLM
NVIDIA Nemotron Team and vLLM Team
- Efficient Decode Context Parallelism with vLLM for Long Context Workloads
Seonghee Lee, Sungsoo Ha, Omri Almog (NVIDIA), Lucas Wilkinson (Red Hat AI)
- vLLM Reaches 25K Total TPS/GPU on Qwen3.5
vLLM Team
- Optimizing vLLM on Arm CPUs
Arm Team
- Parallel All the Way Down: Beyond Single-Token Generation with Speculative Decoding
Alexandre Marques, Megan Flynn, Helen Zhao, Krishna Teja Chitty Venkata, Chibueze Ukachi (Red Hat AI)
- Kimi K3 Is Here: Efficient Day-0 Support on vLLM
vLLM Team and Inferact
- From Day 0 to Production SLAs: Serving GLM-5.2 on 24 NVIDIA B300 GPUs with vLLM
DaoCloud Team
- Announcing vLLM AFD Plugin: Disaggregating Attention and FFN for Flexible MoE Serving
AFD Plugin Contributors
- A Preview of Production-Scale Kimi K3 Support on vLLM
vLLM Team
- Beyond a Single Model: Building Mixture-of-Models Systems with vLLM Semantic Router
vLLM Semantic Router Team
- Keeping vLLM Production Quality: A Look Inside CI, Benchmarking, and the Release Process
Kevin Luu (Inferact)
- TML Inkling on vLLM: Day-0 Support with Optimized Performance
vLLM Team
- vLLM x TileRT: Specialized Decode for Latency-Critical Serving
TileRT team
- EAGLE3 Speculative Decoding on AMD Instinct GPUs: Training and Serving with vLLM and AMD Quark
Larry Li, Chao Li, Haichen Zhang, Chun Fang, Andy Luo, Spandan Tiwari, and Ashish Sirasao
- vime + ROCm: End-to-End RL Post-Training on AMD Instinct™ GPUs
AMD contributors & vime community
- vLLM × HPC-Ops: High-Performance Attention and MoE Backends from Tencent Hunyuan
Tencent Hunyuan AI Infra Team and vLLM Team
- Experience and Lessons Learned from Serving Multi-Stage Qwen3-Omni in vLLM-Omni
vLLM-Omni Team and Ant Group SCT Team
- Micro-Agent: Beat Frontier Models with Collaboration inside Model API
vLLM Semantic Router Team
- Engineering TTS Inference in vLLM-Omni
vLLM-Omni TTS Team
- Beyond One Model: Fusion in vLLM Semantic Router
vLLM Semantic Router Team
- MiniMax M3 in vLLM: Day-0 Serving for 1M-Token Multimodal Reasoning
vLLM Team
- DiffusionGemma: The First Diffusion LLM (dLLM) Natively Supported in vLLM
The vLLM Team and Google DeepMind Team
- Announcing vime: A Simple, Stable, and Efficient RL Framework for LLMs
vime Contributors and the vLLM Team
- vLLM Semantic Router v0.3 Themis: From Signals to Stateful Production Routing
vLLM Semantic Router Team
- Announcing Day-0 Support for NVIDIA Nemotron 3 Ultra on vLLM
NVIDIA Nemotron Team
- Fast & Efficient LLM Inference with vLLM: A New Course with DeepLearning.AI
Cedric Clyburn
- Session-Aware Agentic Routing: Continuity-Aware Model Selection for Long-Horizon LLM Agents
Xunzhuo Liu, Bowei He, Huamin Chen, Haichen Zhang (AMD), Andy Luo (AMD), and the vLLM Semantic Router Team
- Accelerating vLLM-Omni Inference with AutoRound Quantization
vLLM-Omni Community, Intel AutoRound Team
- vLLM on the DGX Spark: Architecture, Configuration, and Local Evaluation
Inferact
- Accelerating Laguna XS.2 Inference with vLLM, Speculators, and LLM Compressor
Megan Flynn, Dipika Sikka, Alexandre Marques
- Native RL APIs in vLLM
Aaron Hao, Sumanth Hegde, Kyle Sayers, Kourosh Hakhamaneshi, and the vLLM team
- Speculators v0.5.0: DFlash Support and Online Training
Fynn Schmitt-Ulms, Helen Zhao, Rahul Tuli and Dipika Sikka (Red Hat AI Model Optimization Team)
- From Text to Multimodal Routing: Hardening Vision Signals in vLLM Semantic Router
David Shrader, Huamin Chen, Xunzhuo Liu, Bowei He, and the vLLM Semantic Router Team
- EAGLE 3.1: Advancing Speculative Decoding Through Collaboration Between the EAGLE Team, vLLM, and TorchSpec
EAGLE Team, vLLM Team, and TorchSpec Team
- vLLM x Novita AI: PegaFlow for Production-Grade External KV Cache
Novita AI and the vLLM Team
- Elastic Expert Parallelism in vLLM
Itay Alroy (NVIDIA), Yongji Wu (Sky Computing), Rui Qiao (Anyscale), Tyler Michael Smith (Red Hat), Moein Khazraee (NVIDIA), Omri Kahalon (NVIDIA), Tzu-Ling Kan (NVIDIA), Ron Tourgeman (NVIDIA)
- Announcing VeRL-Omni: Easy, Fast, and Stable RL Training for Diffusion and Omni-Modality Models
VeRL-Omni Team
- A First Comprehensive Study of TurboQuant: Accuracy and Performance
Eldar Kurtić, Michael Goin, Alexandre Marques (Red Hat AI)
- vLLM Tops the Artificial Analysis Leaderboard
vLLM Team
- Serving Agentic Workloads at Scale with vLLM x Mooncake
Yifan Qiao, Trong Dao Le, Ao Shen, Zhewen Li, Bowen Wang
- Run Highly Efficient Multimodal Agentic AI with NVIDIA Nemotron 3 Nano Omni Using vLLM
NVIDIA Nemotron Team
- DeepSeek V4 in vLLM: Efficient Long-context Attention
vLLM Team
- The State of FP8 KV-Cache and Attention Quantization in vLLM
Jonas Kübler* (AWS), Eldar Kurtić* (Red Hat AI), Lucas Wilkinson (Red Hat AI), Matthew Bonanni (Red Hat AI), Michael Goin (Red Hat AI), Alexandre Marques (Red Hat AI), Kailash Budhathoki (AWS) (* Equal Contribution)
- Disaggregated Serving for Hybrid SSM Models in vLLM
Nicolò Lucchesi, Zhanqiu Hu (Red Hat), and the vLLM team
- vLLM Korea Meetup 2026 Wrap-Up
vLLM Team
- Next-Level Inference: Why Your Single-Node vLLM Setup Needs Prefill-Decode Disaggregation
AMD and Embedded LLM
- Announcing Gemma 4 on vLLM: Byte for byte, the most capable open models
Google Team
- Extracting hidden states from vLLM
Fynn Schmitt-Ulms
- Model Runner V2: A Modular and Faster Core for vLLM
vLLM Team
- P-EAGLE: Faster LLM inference with Parallel Speculative Decoding in vLLM
Amazon and NVIDIA Team
- Run Highly Efficient and Accurate Multi-Agent AI with NVIDIA Nemotron 3 Super Using vLLM
NVIDIA Nemotron Team
- vLLM Semantic Router v0.2 Athena: ClawOS, Model Refresh, and the System Brain
vLLM Semantic Router Team
- vLLM Triton Attention Backend Deep Dive
vLLM Team at IBM Research
- Beyond Porting: How vLLM Orchestrates High-Performance Inference on AMD ROCm
AMD and Embedded LLM
- Efficiently serve dozens of fine-tuned models with vLLM on Amazon SageMaker AI and Amazon Bedrock
Danielle Maddix Robinson, Florian Saupe, George Novack, Haipeng Li, Mani Kumar Adari, Xiang Song, Yu Gong (AWS AI Team)
- DeepSeek-V3.2 on GB300: Performance Breakthrough
The DaoCloud and vLLM team
- Driving vLLM WideEP and Large-Scale Serving Toward Maturity on Blackwell (Part I)
Meta and NVIDIA Team
- GPT-OSS Performance Optimizations on NVIDIA Blackwell: Pushing the Pareto Frontier
The vLLM and NVIDIA team
- Streaming Requests & Realtime API in vLLM
Meta, Mistral AI as well as the vLLM team
- Building Mixture-of-Models on AMD GPUs with vLLM-SR
The AMD and vLLM Semantic Router Team
- Inside vLLM’s New KV Offloading Connector: Smarter Memory Transfer for Maximizing Inference Throughput
Or Ozeri, Danny Harnik (vLLM Team at IBM Research)
- vLLM Semantic Router v0.1 Iris: The First Major Release
vLLM Semantic Router Team
- Introducing vLLM Playground: A Modern Web Interface for Managing and Interacting with vLLM Servers
micytao
2025(51)
- Announcing vllm.ai Website and Some Community Updates
vLLM Team
- vLLM-Omni Diffusion Cache Acceleration
vLLM-Omni Team
- vLLM Large Scale Serving: DeepSeek @ 2.2k tok/s/H200 with Wide-EP
vLLM Team
- AMD × vLLM Semantic Router: Building the System Intelligence Together
The AMD and vLLM Semantic Router Team
- Run Highly Efficient and Accurate AI Agents with NVIDIA Nemotron 3 Nano on vLLM
NVIDIA Nemotron Team
- Encoder Disaggregation for Scalable Multimodal Model Serving
Multimodality Workstream @ vLLM
- Token-Level Truth: Real-Time Hallucination Detection for Production LLMs
vLLM Semantic Router Team
- Diving into speculative decoding training support for vLLM with Speculators v0.3.0
Fynn Schmitt-Ulms, Helen Zhao, Rahul Tuli and Dipika Sikka (Red Hat AI Model Optimization Team)
- vLLM Router: A High-Performance and Prefill/Decode Aware Load Balancer for Large-scale Serving
vLLM Team
- Advancing Low‑Bit Quantization for LLMs: AutoRound x LLM Compressor
Intel Neural Compressor Team, Red Hat AI Model Optimization Team
- Tracing Hanging and Complicated GPU Kernels Down To The Source Code
Kaichao You (vLLM)
- Announcing vLLM-Omni: Easy, Fast, and Cheap Omni-Modality Model Serving
vLLM-Omni Team
- Streamlined multi-node serving with Ray symmetric-run
Richard Liaw (Anyscale/Ray), Kaichao You (vLLM)
- Building Clean, Maintainable vLLM Modifications Using the Plugin System
Dhruvil Bhatt (AWS SageMaker)
- Docker Model Runner Integrates vLLM for High-Throughput Inferencing
Docker Team
- Signal-Decision Driven Architecture: Reshaping Semantic Routing at Scale
vLLM Semantic Router Team
- Shared Memory IPC Caching: Accelerating Data Transfer in LLM Inference Systems
Donglu Wang (Cohere)
- Fast and Affordable LLMs serving on Intel Arc Pro B-Series GPUs with vLLM
Intel vLLM Team
- No More Train-Inference Mismatch: Bitwise Consistent On-Policy Reinforcement Learning with vLLM and TorchTitan
vLLM and TorchTitan Teams
- Run Multimodal Reasoning Agents with NVIDIA Nemotron on vLLM
NVIDIA Nemotron Team
- Chasing 100% Accuracy: A Deep Dive into Debugging Kimi K2's Tool-Calling on vLLM
Linian Wang (Peking University)
- From Monolithic to Modular: Scaling Semantic Routing with Extensible LoRA
Ivar Flakstad (Hugging Face), OneZero-Y, Huamin Chen (Red Hat), Xunzhuo Liu (Tencent)
- Zero-Reload Model Switching with vLLM Sleep Mode
Embedded LLM
- Now Serving NVIDIA Nemotron with vLLM
NVIDIA Nemotron Team
- No More Retokenization Drift: Returning Token IDs via the OpenAI Compatible API Matters in Agent RL
The Agent Lightning (AGL) Team
- vLLM TPU: A New Unified Backend Supporting PyTorch and JAX on TPU
Google Team
- SemiAnalysis InferenceMAX: vLLM and NVIDIA Accelerate Blackwell Inference
vLLM Team
- DeepSeek-V3.2-Exp in vLLM: Fine-Grained Sparse Attention in Action
vLLM Team
- The First vLLM Meetup in Korea
vLLM Team
- vLLM Now Supports Qwen3-Next: Hybrid Architecture with Extreme Efficiency
The vLLM Team
- vLLM Semantic Router: Next Phase in LLM inference
vLLM Semantic Router Team
- Inside vLLM: Anatomy of a High-Throughput LLM Inference System
Aleksa Gordic
- Serving Geospatial, Vision, and Beyond: Enabling Multimodal Output Processing in vLLM
Christian Pinto (IBM Research Europe - Dublin), Michele Gazzetti (IBM Research Europe - Dublin), Michael Johnston (IBM Research Europe - Dublin), Maximilien Philippe Marie de Bayser (IBM Research - Brazil)
- Introduction to torch.compile and How It Works with vLLM
Luka Govedič (Red Hat), Richard Zou (Meta), Addie Stevens (Red Hat), Kaichao You (Tsinghua University), Michael Goin (Red Hat), Saša Zelenović (Red Hat)
- GLM-4.5 Meets vLLM: Built for Intelligent Agents
Yuxuan Zhang
- CUDA Core Dump: An Effective Tool to Debug Memory Access Issues and Beyond
Kaichao You
- vLLM Now Supports gpt-oss
The vLLM Team
- MiniMax-M1 Hybrid Architecture Meets vLLM: Long Context, Fast Inference
MiniMax
- Introducing vLLM Hardware Plugin, Best Practice from Ascend NPU
The Ascend Team on vLLM
- Accelerating RLHF with vLLM, Best Practice from OpenRLHF
The OpenRLHF Team
- Transformers modeling backend integration in vLLM
The Hugging Face Team
- Llama 4 in vLLM
The vLLM Team
- PTPC-FP8: Boosting vLLM Performance on AMD ROCm
AMD and Embedded LLM
- Introducing AIBrix: A Scalable, Cost-Effective Control Plane for vLLM
AIBrix Team
- Distributed Inference with vLLM
vLLM Team
- Introducing vLLM Inference Provider in Llama Stack
Yuan Tang (Red Hat) and Ashwin Bharambe (Meta)
- vLLM V1: A Major Upgrade to vLLM's Core Architecture
vLLM Team
- High Performance and Easy Deployment of vLLM in K8S with vLLM production-stack
LMCache Team
- Structured Decoding in vLLM: a gentle introduction
Guest Post by BentoML and Red Hat
- Installing and Developing vLLM with Ease
vLLM Team
- vLLM 2024 Retrospective and 2025 Vision
vLLM Team
2024(5)
- Serving LLMs on AMD MI300X: Best Practices
Guest Post by Embedded LLM and Hot Aisle Inc.
- How Speculative Decoding Boosts vLLM Performance by up to 2.8x
vLLM Team
- vLLM v0.6.0: 2.7x Throughput Improvement and 5x Latency Reduction
vLLM Team
- vLLM’s Open Governance and Performance Roadmap
vLLM Team
- Announcing Llama 3.1 Support in vLLM
vLLM Team