Rollout-Time Intervention tests agent training on new tasks
The framework adds multi-agent correction to training rollouts and filters for distribution shift.
Event / data period: 2026-10-09
Published: · Updated:
Researchers affiliated with Alibaba and the Hong Kong University of Science and Technology introduced Rollout-Time Intervention (RTI). During training, RTI applies multi-agent corrections to failed attempts using task-specific rubrics, then filters intervention traces that shift too far from the model’s distribution and reweights the retained data.
On the out-of-distribution Claw-Eval-Hard20 test, the paper reports a Pass@1 rate of 62.92% for the model trained with RTI. The task families in this evaluation were not used in synthetic training, but the experiment remains limited to researcher-created tasks and API simulations; the paper also reports an in-distribution result tied to the same task families. The findings do not establish general-purpose agent performance.
Data
Research result
Claw-Eval-Hard20; task families excluded from training synthesis; Pass@1
| RTI model Pass@1 on Claw-Eval-Hard20 out-of-distribution tasks | 62.92 | Claw-Eval-Hard20; task families excluded from training synthesis; Pass@1 |
Methodology
The value is limited to the benchmark experiment reported in the paper; it is not a field result or general performance guarantee.
Updated:
Subjects and places
Subject proposals
- Science and technologySuggested
- Preserved editorial category/topic; broad candidate onlyarxiv.org ↗
Geographic references
Only source-provided geographic references are shown. Opening an external map is optional. Optional story maps use OpenStreetMap. Opening a map sends your IP address and this site’s address to the map provider; location data remains readable as text without loading the map.
- China · Place role: Coverage · Place type: Country or area · Location precision: RegionalEvidence
- Explicit preserved editorial country/region; no city or national centroid inferredarxiv.org ↗