RadixArk with Annapurna Labs (AWS) · Trainium inference plugin

SGLang-Neuron

SGLang-Neuron is an out-of-tree plugin that lets SGLang serve models and generate RL rollouts on AWS Trainium using the existing SGLang interface. It runs on native PyTorch, uses Neuron Kernel Interface kernels, and supports prefill-decode disaggregation with KV-cache transfer over EFA. It is available through an AWS Deep Learning Container, and the page gives a launch example serving openai/gpt-oss-120b on a Trainium3 instance.