Deploy frontier AI models on AWS Trainium with SGLang-Neuron
Amazon announced public availability of SGLang-Neuron, delivered through an AWS Deep Learning Container and released by RadixArk in collaboration with Annapurna Labs. The out-of-tree plugin lets users serve models and generate RL rollouts on AWS Trainium through the existing SGLang server interface and generation APIs. It runs on native PyTorch, uses NKI kernels compiled with torch.compile, and supports PD disaggregation with KV-cache transfer via NIXL over EFA. The post includes a walkthrough for launching a container and serving openai/gpt-oss-120b on a Trainium3 EC2 instance.
Why it matters: It gives AI engineers a concrete, documented path to run the familiar SGLang serving workflow on AWS Trainium, including deployment steps and supported architecture details.