Newsverum.
SearchK

Search stories

Find the story. Follow the source.

Explore
Explore the data1840Agriculture232Business287Climate46Culture12Economy264Education60Energy427Environment44Finance62Fiscal Policy46Government51Health96Infrastructure88Politics18Science25Security33Society14Sports29Technology310
EN
العربيةDeutschEnglishEspañolFrançaisPortuguêsTürkçe
Sections
Technology / WORLD

AWS reports MoE kernel optimization results on SageMaker

AWS detailed GPU kernel optimizations for mixture-of-experts models on SageMaker AI on October 13. The company described swapping matrix operands, routing FP8 operations to the Triton backend and removing unnecessary just-in-time recompilations.

Newsverum · Source brief1 min read

Source review period: 2026-10-13

Published: · Updated:

The source does not state the event date; the period shown is the source review period.

Reading sizeSources ↓Share story ↗
Conceptual illustration of technology, processors and connected information.Illustration
Symbolic illustration

AWS detailed GPU kernel optimizations for mixture-of-experts models on SageMaker AI on October 13. The company described swapping matrix operands, routing FP8 operations to the Triton backend and removing unnecessary just-in-time recompilations.

AWS reports up to 10% higher output throughput and up to 9% lower time to first token on H200 GPUs. These are company measurements on particular configurations, not guarantees for every model. The report places the Triton routing change in LMI v28 and the recompilation fix in vLLM 0.22.0 or later.

Data

Timeline

AWS reports MoE kernel optimization results on SageMaker

AWS reports MoE kernel optimization results on SageMaker

  1. · Occurred

    Publication of the research report

    AWS detailed GPU kernel optimizations for mixture-of-experts models on SageMaker AI on October 13. The company described swapping matrix operands, routing FP8 operations to the Triton backend and removing unnecessary just-in-time recompilations.

Methodology

Reported research results are attributed to the source; no independent performance measurement was made.

Updated:

DownloadCSV ↓JSON ↓
Table

AWS reports MoE kernel optimization results on SageMaker

AWS reports MoE kernel optimization results on SageMaker

Output throughput increase10
Time to first token reduction9
Methodology

Maximum improvements reported by AWS for particular MoE configurations on H200. The first-token correction was measured at eight-way tensor parallelism; percentages are not added together.

Updated:

DownloadCSV ↓JSON ↓
Structured coverage

Subjects and places

Subject proposals

Geographic references

Only source-provided geographic references are shown. Opening an external map is optional. Optional story maps use OpenStreetMap. Opening a map sends your IP address and this site’s address to the map provider; location data remains readable as text without loading the map.

  • United States · Place role: Coverage · Place type: Country or area · Location precision: Regional
    Evidence
    • Explicit preserved editorial country/region; no city or national centroid inferredaws.amazon.com ↗
Taxonomy version: IPTC Media Topics 2026-Q2

Continue reading

Technology

Microsoft reports completion of MAI-Base-2 pre-training

Microsoft AI said on October 15 that it completed pre-training MAI-Base-2, a mixture-of-experts model with 2.8 trillion total and 80 billion active parameters. The company says training used 16,000 GB200 GPUs over 38 days in August and September; the model is still undergoing post-training.

Microsoft AI1 min read
Data included
Technology

AWS details labsweep measurements for AI training sandboxes

AWS published a technical evaluation on October 9 of reinforcement-learning rollouts in Firecracker microVMs. It reported more than 100,000 samples across over 500 configurations using the labsweep measurement harness. In the tested workloads, tail latency rose sharply beyond one microVM per virtual CPU, while median latency changed much less.

AWS Compute Blog1 min read
Data included