Live intelligence

Latest AI updates

1 updates
Products Editor selected

Buoyant Releases Kinference, an Open-Source Inference Control Plane for Kubernetes

Buoyant has released Kinference v0.1.0-alpha.7, an Apache 2.0 control plane that routes requests across vLLM workers on Kubernetes using declarative policies, queue state and KV-cache signals. Self-reported tests showed substantial latency and goodput improvements over random Kubernetes Service routing for shared-prefix and multiturn workloads, but little median improvement for prompts without shared prefixes. The project remains alpha software, with advertised features including token rate limiting, semantic classifiers and Anthropic API support not yet implemented.

Why it matters: Kinference offers AI infrastructure teams a concrete new option for policy-driven, multi-tenant vLLM routing, while its alpha status and limited benchmark baseline warrant caution.

BuoyantKinferenceKubernetes inference