# AWS reports MoE kernel optimization results on SageMaker

> AWS detailed GPU kernel optimizations for mixture-of-experts models on SageMaker AI on October 13. The company described swapping matrix operands, routing FP8 operations to the Triton backend and removing unnecessary just-in-time recompilations.

- Canonical URL: https://newsverum.com/en/news/aws-sagemaker-moe-kernel-optimization-report-2026
- Publisher: Newsverum
- Section: Technology
- Country: United States
- Published: 2026-10-15T20:48:36+00:00
- Updated: 2026-10-15T20:48:36+00:00
- Source review period: 2026-10-13
- Other editions: [Türkçe](https://newsverum.com/tr/news/aws-sagemaker-moe-kernel-optimization-report-2026.md), [العربية](https://newsverum.com/ar/news/aws-sagemaker-moe-kernel-optimization-report-2026.md), [Français](https://newsverum.com/fr/news/aws-sagemaker-moe-kernel-optimization-report-2026.md), [Español](https://newsverum.com/es/news/aws-sagemaker-moe-kernel-optimization-report-2026.md), [Deutsch](https://newsverum.com/de/news/aws-sagemaker-moe-kernel-optimization-report-2026.md), [Português](https://newsverum.com/pt/news/aws-sagemaker-moe-kernel-optimization-report-2026.md)

*The source does not state the event date; the period shown is the source review period.*

AWS detailed GPU kernel optimizations for mixture-of-experts models on SageMaker AI on October 13. The company described swapping matrix operands, routing FP8 operations to the Triton backend and removing unnecessary just-in-time recompilations.

AWS reports up to 10% higher output throughput and up to 9% lower time to first token on H200 GPUs. These are company measurements on particular configurations, not guarantees for every model. The report places the Triton routing change in LMI v28 and the recompilation fix in vLLM 0.22.0 or later.

## Data

### AWS reports MoE kernel optimization results on SageMaker

AWS reports MoE kernel optimization results on SageMaker

Methodology: Reported research results are attributed to the source; no independent performance measurement was made.

| date | label | description | status |
|---|---|---|---|
| 2026-10-13 | Publication of the research report | AWS detailed GPU kernel optimizations for mixture-of-experts models on SageMaker AI on October 13. The company described swapping matrix operands, routing FP8 operations to the Triton backend and removing unnecessary just-in-time recompilations. | occurred |

Sources: <https://aws.amazon.com/blogs/machine-learning/mixture-of-experts-kernel-engineering-on-amazon-sagemaker-ai/> · Download data: [JSON](https://newsverum.com/api/v1/articles/aws-sagemaker-moe-kernel-optimization-report-2026/data/research-publication?lang=en&format=json) · [CSV](https://newsverum.com/api/v1/articles/aws-sagemaker-moe-kernel-optimization-report-2026/data/research-publication?lang=en&format=csv)

### AWS reports MoE kernel optimization results on SageMaker

AWS reports MoE kernel optimization results on SageMaker

Methodology: Maximum improvements reported by AWS for particular MoE configurations on H200. The first-token correction was measured at eight-way tensor parallelism; percentages are not added together.

| Indicator | Reported maximum improvement |
|---|---|
| Output throughput increase | 10 |
| Time to first token reduction | 9 |

Sources: <https://aws.amazon.com/blogs/machine-learning/mixture-of-experts-kernel-engineering-on-amazon-sagemaker-ai/> · Download data: [JSON](https://newsverum.com/api/v1/articles/aws-sagemaker-moe-kernel-optimization-report-2026/data/reported-performance-changes?lang=en&format=json) · [CSV](https://newsverum.com/api/v1/articles/aws-sagemaker-moe-kernel-optimization-report-2026/data/reported-performance-changes?lang=en&format=csv)

## Sources

- [AWS Artificial Intelligence Blog](<https://aws.amazon.com/blogs/machine-learning/mixture-of-experts-kernel-engineering-on-amazon-sagemaker-ai/>) (Source checked: 2026-10-15; Published: 2026-10-13)

---

Cite the canonical URL above. [Editorial standards and corrections](https://newsverum.com/en/editorial)