Cohere study compares coding agent team strategies
In Cohere’s ProgramBench tests, planning-focused teams reached similar scores at lower reported cost.
Event / data period: 2026-10-14
Published: · Updated:
Cohere researchers published a controlled study comparing ways to organize coding-agent teams. On 200 ProgramBench tasks, an asynchronous GPT-5.6 Sol team delegating only planning reported 63.6% reward at a cost of $249; a GPT-5.6 Sol team delegating all stages scored 63.2% at $510. Both figures were averages across three runs.
A GPT-5.6 Sol lead coordinating 16 GPT-5.6 Luna planning workers scored 61.8% at $155, compared with 63.6% at $249 for the all-Sol planning team. This fixed 16-worker configuration was run once. Cohere says the findings are specific to its test harness; some model comparisons used a single trial, and the results do not establish general agent performance.
Data
Cohere study compares coding agent team strategies
Date and scope of the development reported in the primary source.
- · Occurred
Cohere study compares coding agent team strategies
In Cohere’s ProgramBench tests, planning-focused teams reached similar scores at lower reported cost.
Methodology
The announcement date published by the source is recorded as one event.
Updated:
Cohere ProgramBench team results
ProgramBench results from Cohere’s coding-agent team evaluation; reward and inference cost are reported as stated by the source.
| GPT-5.6 Sol; async lead delegates planning only | 63.6 | 249 | 200 tasks; mean of 3 runs |
| GPT-5.6 Sol; async lead delegates all stages | 63.2 | 510 | 200 tasks; mean of 3 runs |
| GPT-5.6 Sol lead; 16 GPT-5.6 Luna planning workers | 61.8 | 155 | Fixed 16-worker configuration; 1 run |
Methodology
Reward is the mean share of hidden behavioral tests passed across 200 ProgramBench tasks. The first two asynchronous GPT-5.6 Sol configurations average three runs; the fixed configuration with 16 GPT-5.6 Luna workers was run once. Results are from Cohere’s own test harness.
Updated:
Subjects and places
Subject proposals
- Science and technologySuggested
- Preserved editorial category/topic; broad candidate onlycohere.com ↗
Geographic references
Only source-provided geographic references are shown. Opening an external map is optional. Optional story maps use OpenStreetMap. Opening a map sends your IP address and this site’s address to the map provider; location data remains readable as text without loading the map.
- Canada · Place role: Coverage · Place type: Country or area · Location precision: RegionalEvidence
- Explicit preserved editorial country/region; no city or national centroid inferredcohere.com ↗