Ai2 details scheduler that changes how its GPU clusters share capacity

The US-based Allen Institute for AI (Ai2) replaced priority-based scheduling across its research clusters with a system that assigns GPU-time budgets to teams and projects, allocates capacity through hierarchical fair-share rules and can requeue jobs after declared minimum runtimes. The institute said its clusters contain 88 to 1,024 GPUs and that roughly 150 researchers submit workloads requesting two to three times the available capacity at any moment.

During the 30-day period after a rollout that began at the end of July, teams received 98% of the GPU hours owed to them based on demand and allocations; cluster occupancy remained at 98% before and after the change. Ai2 also reported a 74% reduction in repairs requiring human intervention and a fall in median queue wait on its largest H100 cluster from five minutes to 24 seconds. It said it is still monitoring issues including interrupted interactive sessions and possible capacity fragmentation for large jobs.