CompileMask improves throughput for custom transformer attention masks
Researchers at JD.com and Beijing University of Posts and Telecommunications developed CompileMask, a compiler that connects flexible attention masks to efficient GPU kernels. Blackwell GPU experiments measured higher throughput than Flex-FA4 for several mask patterns.
Source review period: 2026-10-09
Published: · Updated:
The source does not state the event date; the period shown is the source review period.
Transformer workloads use different masks to control which tokens can attend to one another. CompileMask lets users define rules in a region-based language; its compiler derives visible token intervals ahead of execution and supplies a compact descriptor to the GPU attention kernel at runtime.
On NVIDIA Blackwell at sequence length 32K, the study reports throughput over Flex-FA4 of 1.44× for HSTU, 2.13× for a shared-prompt reward model and 2.87× for CLEAR. For causal attention it retained 85–89% of native FA4 throughput. These comparisons cover specific masks and hardware settings and do not establish the same gain for every transformer workload.
Data
Attention-mask comparison
Attention workloads measured at length 32K; values are specific to the study.
| Causal attention, lower bound | 85 |
| Causal attention, upper bound | 89 |
Methodology
Four mask patterns were tested on NVIDIA Blackwell GPUs at sequence lengths from 1K to 32K; speedups are relative to the paper’s Flex-FA4 baseline.
Updated:
Subjects and places
Subject proposals
- Telecommunication serviceSuggested
- Subject suggested from the title or summary: Telecommunicationsarxiv.org ↗
- Science and technologySuggested
- Preserved editorial category/topic; broad candidate onlyarxiv.org ↗
Geographic references
Only source-provided geographic references are shown. Opening an external map is optional. Optional story maps use OpenStreetMap. Opening a map sends your IP address and this site’s address to the map provider; location data remains readable as text without loading the map.
- China · Place role: Coverage · Place type: Country or area · Location precision: RegionalEvidence
- Explicit preserved editorial country/region; no city or national centroid inferredarxiv.org ↗