# SlackSwap combines data transfers and recomputation in GPU-limited training

> Researchers at UC Merced and the University of Texas at Austin introduced SlackSwap, a planner for deep-learning training. It recomputes some activations when the transfer budget between a GPU and host memory is exceeded; reported results come from simulation, not hardware measurements.

- Canonical URL: https://newsverum.com/en/news/slackswap-activation-memory-gpu-training-2026
- Publisher: Newsverum
- Section: Technology
- Country: United States
- Published: 2026-10-13T01:40:12+00:00
- Updated: 2026-10-13T01:40:12+00:00
- Source review period: 2026-10-09
- Other editions: [Türkçe](https://newsverum.com/tr/news/slackswap-activation-memory-gpu-training-2026.md), [العربية](https://newsverum.com/ar/news/slackswap-activation-memory-gpu-training-2026.md), [Français](https://newsverum.com/fr/news/slackswap-activation-memory-gpu-training-2026.md), [Español](https://newsverum.com/es/news/slackswap-activation-memory-gpu-training-2026.md), [Deutsch](https://newsverum.com/de/news/slackswap-activation-memory-gpu-training-2026.md), [Português](https://newsverum.com/pt/news/slackswap-activation-memory-gpu-training-2026.md)

*The source does not state the event date; the period shown is the source review period.*

Intermediate activations saved for backpropagation can exceed GPU memory during neural-network training. SlackSwap profiles a training step and treats the transfer time that can be hidden during computation as a budget: it moves activations that are costly to rebuild within that budget and recomputes the rest.

In a discrete-event simulation parameterized for an A100 and PCIe 4.0, with the activation-memory budget set to half the no-memory-limit GPU’s peak, SlackSwap retained 96.3% of ideal throughput for BERT-Large, 97.6% for GPT-2 XL and 99.6% for a Llama-7B-style model. The authors say all results are simulated and that effects such as allocator fragmentation are not modeled; these are not measurements from physical GPU training.

## Data

### Simulated training throughput

Results are from simulation, not a run on a physical GPU.

Methodology: The study evaluates three transformer workloads at a 50% activation-memory budget in a discrete-event simulation parameterized for an A100 and PCIe 4.0.

| Model | Throughput relative to ideal |
|---|---|
| BERT-Large | 96.3 |
| GPT-2 XL | 97.6 |
| Llama-7B-style | 99.6 |

Sources: <https://arxiv.org/abs/2610.13003> · Download data: [JSON](https://newsverum.com/api/v1/articles/slackswap-activation-memory-gpu-training-2026/data/simulated-training-throughput?lang=en&format=json) · [CSV](https://newsverum.com/api/v1/articles/slackswap-activation-memory-gpu-training-2026/data/simulated-training-throughput?lang=en&format=csv)

## Sources

- [arXiv](<https://arxiv.org/abs/2610.13003>) (Source checked: 2026-10-13; Published: 2026-10-09)

---

Cite the canonical URL above. [Editorial standards and corrections](https://newsverum.com/en/editorial)