Accelerating vLLM-Omni Inference with AutoRound Quantization
·10 min read
We are excited to announce that AutoRound — Intel's state-of-the-art post-training quantization (PTQ) algorithm — is now fully integrated into vLLM-Omni, enabling a streamlined quantize-once,...