Latest AI updates

SageMaker Inference now supports Multi-Instance GPU to maximize GPU utilization for model endpoints

AWS SageMaker Inference now supports NVIDIA Multi-Instance GPU (MIG) partitioning, allowing a single physical GPU to be divided into isolated partitions for deploying multiple models or right-sizing capacity. This feature works with inference components, enabling hardware-enforced isolation, predictable latency, and no cold starts. Benchmarks show a partition of an H100 outperforms a full A100 in throughput and cost for many workloads.