Live intelligence

Latest AI updates

1 updates
Products Editor selected

Amazon SageMaker HyperPod Global Inference Gateway routes across clusters and Regions

AWS introduced Amazon SageMaker HyperPod Global Inference Gateway, a SageMaker-managed endpoint that routes inference requests across registered HyperPod clusters in one Region or across Regions. The gateway scores healthy clusters using KV cache headroom, queue depth, Region proximity and error rate, with configurable routing weights, and automatically moves traffic away from failed clusters. Requests are authenticated with SigV4 and IAM and travel over AWS PrivateLink; the feature is available in 13 AWS Regions.

Why it matters: It gives AI platform teams a concrete AWS-managed option for cross-cluster and cross-Region inference routing with automatic failover, reducing custom load-balancing and networking work.

Amazon SageMaker HyperPodAWSAI inference