Introducing Kinference: An Open Source Inference Control Plane for vLLM and Kubernetes
·7 min read
Kinference is a new open source inference control plane for building scalable inference platforms with vLLM and Kubernetes, providing token rate limiting, tenant usage and cost tracking, intelligent load balancing, and inference policies.