llm-d 1.0 Ships as First Stable Release of Open-Source Distributed Inference Stack
Google and its collaborators released llm-d 1.0, the first stable version of the open-source, Kubernetes-native distributed inference stack that pools vLLM or SGLang model servers. The release adds multi-tenant flow cont...