Live intelligence

Latest AI updates

3 updates
Products Editor selected

Amazon SageMaker HyperPod adds workload recovery and expanded infrastructure checks

AWS introduced three capabilities for Amazon SageMaker HyperPod: the MultiNodeTrainingValidation deep health check, which runs a small Megatron-LM training workload on each instance and then across instances to catch GPUs that pass hardware tests but produce NaN or infinite loss; a Kubernetes Resource Monitoring Agent for EKS-orchestrated clusters that reports device plugin crash loops, missing DaemonSets, node condition and taint changes, and the pods and nodes behind failed training jobs; and hung job detection for Slurm clusters that flags stalled jobs and can cancel them so Slurm requeues them. AWS frames the features as addressing numerical faults, hung jobs and cluster-level failures that node-level health monitoring misses. On a two-node test cluster, the two MultiNodeTrainingValidation tests added about eight minutes to a full deep health check that took roughly 1 hour 28 minutes.

Why it matters: Teams running large distributed training on HyperPod get concrete new failure-detection and recovery features, with setup steps and reported results, that target wasted GPU time from silent corruption, hung jobs and Kubernetes-level faults.

Amazon SageMaker HyperPodAWSDistributed training reliability
Products Editor selected

Amazon SageMaker HyperPod Inference adds managed NVIDIA Dynamo support

AWS now supports NVIDIA Dynamo on SageMaker HyperPod Inference, bringing multi-node deployment, disaggregated prefill/decode and KV-aware routing to managed LLM endpoints. The HyperPod Inference add-on installs Dynamo's operators and scheduling components, with configuration available either through a bring-your-own DynamoGraphDeployment or an InferenceEndpointConfig with a dynamoSpec block. The post walks through aggregated and disaggregated GLM-5.3 deployments on ml.p5.48xlarge instances and reports benchmarks: disaggregated serving reached 2.05x the highest aggregated throughput for Qwen3-235B-A22B-FP8, while a two-node aggregated deployment retained 99.1 percent of single-node per-GPU throughput. It closes with tuning guidance covering recipes, EFA verification and AIPerf-based benchmarking.

Why it matters: Gives teams running large models a concrete, documented path to distributed and disaggregated inference on managed AWS infrastructure, including deployment configs and measured throughput trade-offs.

Amazon SageMaker HyperPodNVIDIA DynamoLLM inference infrastructure
Tutorial Editor selected

AWS shows SFT-then-GRPO recipe to make GPT-OSS-20B reason in non-English languages

AWS details a fine-tuning workflow on Amazon SageMaker HyperPod that teaches GPT-OSS-20B to reason and answer in French, German, Spanish and Italian instead of defaulting to English. Supervised fine-tuning with LoRA raised reasoning-language accuracy from 22% to 100% on a 50-case evaluation, while adding GRPO on top of the SFT checkpoint lifted answer-language accuracy from 62% to 70%. A controlled Qwen2.5-7B experiment across multilingual reasoning and GSM8K tasks supports the argument that GRP

Why it matters: It offers a concrete, reproducible fine-tuning methodology and decision framework for choosing between SFT and GRPO when adapting open-weight reasoning models to non-English languages.

Amazon SageMaker HyperPodGPT-OSS-20Breinforcement learning