Texas Tech researchers test reliability-aware tool retrieval for agents
RATED combines semantic matching with feedback on whether tools work during execution.
Event / data period: 2026-10-09
Published: · Updated:
Texas Tech University researchers introduced RATED, a method that ranks agent tools using both query relevance and execution feedback. It pairs a semantic encoder with a contextual bandit that reorders tools after success or failure signals.
On the EcoAgent benchmark, the authors report a 52.2% relative MRR gain over the ToolLLM API retriever. In a simulated outage affecting 30% of tools, the rate of selecting failed tools fell from 86.2% for that baseline to 2.9% with RATED. These are benchmark-query and injected-failure results; they do not establish reliability in production.
Data
Measure
EcoAgent real-query evaluation; compared with ToolLLM API Retriever
| RATED: relative MRR gain | 52.2 | EcoAgent real-query evaluation; compared with ToolLLM API Retriever |
Methodology
Values come from the paper's stated benchmark experiments; they are not field measurements or general performance guarantees.
Updated:
Measure
EcoAgent simulated outage experiment; share of tools
| Share of tools with injected outages | 30 | EcoAgent simulated outage experiment; share of tools |
Methodology
Values come from the paper's stated benchmark experiments; they are not field measurements or general performance guarantees.
Updated:
Measure
ToolLLM API Retriever in the same simulated outage test
| Failed-tool selection with ToolLLM API Retriever | 86.2 | ToolLLM API Retriever in the same simulated outage test |
Methodology
Values come from the paper's stated benchmark experiments; they are not field measurements or general performance guarantees.
Updated:
Measure
RATED in the same simulated outage test
| Failed-tool selection with RATED | 2.9 | RATED in the same simulated outage test |
Methodology
Values come from the paper's stated benchmark experiments; they are not field measurements or general performance guarantees.
Updated:
Subjects and places
Subject proposals
- Science and technologySuggested
- Preserved editorial category/topic; broad candidate onlyarxiv.org ↗
Geographic references
Only source-provided geographic references are shown. Opening an external map is optional. Optional story maps use OpenStreetMap. Opening a map sends your IP address and this site’s address to the map provider; location data remains readable as text without loading the map.
- United States · Place role: Coverage · Place type: Country or area · Location precision: RegionalEvidence
- Explicit preserved editorial country/region; no city or national centroid inferredarxiv.org ↗