{"article":{"annotation":{"article_id":"french-lab-publishes-benchmark-for-long-horizon-robot-agents","available_at":"2026-10-12T23:33:55.270622Z","geography_revision":"ef6ee0010dda5fe7b240fd514749b43e214c57da73222d2e40ec83b276dd0977","id":"da38c4b1171436002147ea99c0a492f7629f0fc19f46eb96a8b7d9b9a8034060","input_sha256":"170151940e25530d7acf9038ef062049bc0ce47c55d7d6034f6b34c5055c5cd5","limitations":["Rule-derived subjects are candidates, not verified editorial labels or calibrated model probabilities.","Coverage is distinct from event location. Unknown country/city/coordinates remain absent; no automatic geocoding.","Article geometries keep their stated precision and methodology; reproducing them is not independent coordinate verification."],"method":"controlled-rules-v1","places":[{"admin1":null,"city":null,"coordinate_note":null,"coordinate_reference_system":"OGC:CRS84","coordinate_source":null,"country":"FR","evidence":[{"field":"country","language":"en","note":"Explicit preserved editorial country/region; no city or national centroid inferred","source_urls":["https://arxiv.org/pdf/2610.13534"]}],"geometry":null,"id":"iso3166-1:FR","kind":"country-or-area","labels":{"ar":"فرنسا","en":"France","es":"Francia","fr":"France"},"match_status":"candidate","precision":"regional","role":"coverage","uncertainty_m":null}],"rules_sha256":"fb5ad3c27c91cb840f7a0a05f7f64a5cb26ad0374409ff9457f277c7092e22b4","schema_version":1,"subjects":[{"concept_id":"medtop:13000000","confidence":null,"confidence_method":null,"evidence":[{"field":"category+topics","language":"en","note":"Preserved editorial category/topic; broad candidate only","source_urls":["https://arxiv.org/pdf/2610.13534"]}],"method":"controlled-rules-v1","rule_id":"technology-broad","status":"candidate"}],"taxonomy_revision":"1f2db31d811fca07e99b16d402c10e2cc2965c9df1c7b41d9be348b40a24e7fb","taxonomy_version":"IPTC Media Topics 2026-Q2"},"api_url":"https://newsverum.com/api/v1/articles/french-lab-publishes-benchmark-for-long-horizon-robot-agents?lang=en","category":"technology","content":{"key_points":[],"paragraphs":["Researchers from LAAS-CNRS and French company Siléane introduced MAGMA-Bench in a preprint dated 9 October. The benchmark evaluates persistent reasoning in long robotic tasks. It includes 12 scenario families with one to three robots, mission changes, interruptions, partial observations and simulated execution failures.","The evaluation contains 1,254 episodes. The authors say even the strongest models they tested fail on a substantial share of clean episodes, with success rates falling further under perturbations. A shared subset of 215 episodes was used to compare frontier models. Because the benchmark separates high-level decisions from perception and low-level control and relies on simulation, the results do not establish the performance of deployed robots."],"sections":[]},"corrections":[],"country":"FR","data":{"blocks":[{"columns":[{"key":"indicator","label":{"ar":"المؤشر","de":"Indikator","en":"Indicator","es":"Indicador","fr":"Indicateur","pt":"Indicador","tr":"Gösterge"},"type":"text"},{"key":"count","label":{"ar":"العدد في نطاق التجربة","de":"Zahl im Versuchsrahmen","en":"Experimental scope count","es":"Cantidad en el alcance experimental","fr":"Nombre dans le périmètre expérimental","pt":"Quantidade no escopo experimental","tr":"Deney kapsamındaki sayı"},"type":"number","unit":"count"},{"key":"released","label":{"ar":"تاريخ نشر المسودة الأولية","de":"Veröffentlichungsdatum des Preprints","en":"Preprint publication date","es":"Fecha de publicación de la prepublicación","fr":"Date de publication de la prépublication","pt":"Data de publicação da pré-publicação","tr":"Ön baskının yayın tarihi"},"type":"date"}],"description":{"ar":"حلقات محاكاة عبر 12 عائلة سيناريو؛ وليست نشر روبوتات حقيقية","de":"Simulierte Episoden aus zwölf Szenariofamilien; kein Einsatz realer Roboter","en":"Simulated episodes across 12 scenario families; not real-robot deployment","es":"Episodios simulados en 12 familias de escenarios; no son despliegues robóticos reales","fr":"Épisodes simulés dans 12 familles de scénarios ; pas un déploiement réel de robots","pt":"Episódios simulados em 12 famílias de cenários; não são implantações reais de robôs","tr":"12 senaryo ailesinde simülasyon bölümleri; gerçek robot dağıtımı değil"},"id":"study-experimental-scope","methodology":{"ar":"العدد الذي أوردته الورقة لنطاق التجربة؛ وليس إحصاءً وطنياً أو نتيجة ميدانية.","de":"Die Zahl bezeichnet den im Artikel genannten Versuchsrahmen; keine nationale Statistik oder Feldmessung.","en":"Count reported by the paper for its experimental scope; not a national statistic or field result.","es":"Cifra del alcance experimental indicado en el artículo; no es una estadística nacional ni un resultado de campo.","fr":"Nombre indiqué par l’article pour le périmètre expérimental ; ce n’est ni une statistique nationale ni un résultat de terrain.","pt":"Número informado no escopo experimental do artigo; não é estatística nacional nem resultado de campo.","tr":"Sayı, kaynak çalışmada bildirilen deney kapsamıdır; ulusal istatistik veya saha sonucu değildir."},"rowTranslations":[{"indicator":{"ar":"حلقات التقييم في MAGMA-Bench","de":"MAGMA-Bench-Bewertungsepisoden","en":"MAGMA-Bench evaluation episodes","es":"Episodios de evaluación de MAGMA-Bench","fr":"Épisodes d’évaluation de MAGMA-Bench","pt":"Episódios de avaliação do MAGMA-Bench","tr":"MAGMA-Bench değerlendirme bölümü"}}],"rows":[{"count":1254,"indicator":"MAGMA-Bench evaluation episodes","released":"2026-10-09"}],"sourceUrls":["https://arxiv.org/pdf/2610.13534"],"title":{"ar":"نطاق الدراسة","de":"Umfang der Studie","en":"Study scope","es":"Alcance del estudio","fr":"Périmètre de l’étude","pt":"Escopo do estudo","tr":"Çalışmanın kapsamı"},"type":"table","updatedAt":"2026-10-12T23:24:06Z"}],"version":1},"data_downloads":[{"formats":{"csv":"https://newsverum.com/api/v1/articles/french-lab-publishes-benchmark-for-long-horizon-robot-agents/data/study-experimental-scope?lang=en&format=csv","json":"https://newsverum.com/api/v1/articles/french-lab-publishes-benchmark-for-long-horizon-robot-agents/data/study-experimental-scope?lang=en&format=json"},"id":"study-experimental-scope"}],"data_types":["table"],"editorial_illustration":{"alt":"Conceptual illustration of technology, processors and connected information.","asset_id":"newsverum-v1-technology","avif_variants":[{"height":320,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-480.avif","width":480},{"height":480,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-720.avif","width":720},{"height":640,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-960.avif","width":960},{"height":800,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1200.avif","width":1200},{"height":1024,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1536.avif","width":1536}],"credit":"Newsverum","documentary":false,"height":1024,"kind":"symbolic-editorial-illustration","language":"en","scope":"topic","url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1536.webp","variants":[{"height":320,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-480.webp","width":480},{"height":640,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-960.webp","width":960},{"height":1024,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1536.webp","width":1536}],"width":1536},"event_id":"event-f10de3bd-56e1-5fcd-bf1f-d0f4b80263f7","event_period":"2026-10-09","event_period_basis":"event-or-reference-period","id":"french-lab-publishes-benchmark-for-long-horizon-robot-agents","image":null,"importance":0.6,"language":"en","modified_at":"2026-10-12T23:33:38+00:00","publication_basis":"publication-record","published_at":"2026-10-12T23:33:38+00:00","reading_minutes":1,"region":"world","section_illustration":{"alt":"Conceptual illustration of technology, processors and connected information.","asset_id":"newsverum-v1-technology","avif_variants":[{"height":320,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-480.avif","width":480},{"height":480,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-720.avif","width":720},{"height":640,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-960.avif","width":960},{"height":800,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1200.avif","width":1200},{"height":1024,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1536.avif","width":1536}],"credit":"Newsverum","documentary":false,"height":1024,"kind":"symbolic-editorial-illustration","language":"en","scope":"section","url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1536.webp","variants":[{"height":320,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-480.webp","width":480},{"height":640,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-960.webp","width":960},{"height":1024,"url":"https://newsverum.com/assets/newsverum/illustrations/v1/technology-1536.webp","width":1536}],"width":1536},"slug":"french-lab-publishes-benchmark-for-long-horizon-robot-agents","sources":[{"checkedAt":"2026-10-13","name":"arXiv preprint by LAAS-CNRS and Siléane researchers","publishedAt":"2026-10-09","url":"https://arxiv.org/pdf/2610.13534"}],"summary":"MAGMA-Bench evaluates agents in simulated multi-robot tasks with changing goals and execution failures.","text":"MAGMA-Bench evaluates agents in simulated multi-robot tasks with changing goals and execution failures.\n\nResearchers from LAAS-CNRS and French company Siléane introduced MAGMA-Bench in a preprint dated 9 October. The benchmark evaluates persistent reasoning in long robotic tasks. It includes 12 scenario families with one to three robots, mission changes, interruptions, partial observations and simulated execution failures.\n\nThe evaluation contains 1,254 episodes. The authors say even the strongest models they tested fail on a substantial share of clean episodes, with success rates falling further under perturbations. A shared subset of 215 episodes was used to compare frontier models. Because the benchmark separates high-level decisions from perception and low-level control and relies on simulation, the results do not establish the performance of deployed robots.","title":"French lab releases benchmark for long-horizon robot agents","topics":[],"translations":[{"api_url":"https://newsverum.com/api/v1/articles/french-lab-publishes-benchmark-for-long-horizon-robot-agents?lang=tr","language":"tr","url":"https://newsverum.com/tr/news/french-lab-publishes-benchmark-for-long-horizon-robot-agents"},{"api_url":"https://newsverum.com/api/v1/articles/french-lab-publishes-benchmark-for-long-horizon-robot-agents?lang=en","language":"en","url":"https://newsverum.com/en/news/french-lab-publishes-benchmark-for-long-horizon-robot-agents"},{"api_url":"https://newsverum.com/api/v1/articles/french-lab-publishes-benchmark-for-long-horizon-robot-agents?lang=ar","language":"ar","url":"https://newsverum.com/ar/news/french-lab-publishes-benchmark-for-long-horizon-robot-agents"},{"api_url":"https://newsverum.com/api/v1/articles/french-lab-publishes-benchmark-for-long-horizon-robot-agents?lang=fr","language":"fr","url":"https://newsverum.com/fr/news/french-lab-publishes-benchmark-for-long-horizon-robot-agents"},{"api_url":"https://newsverum.com/api/v1/articles/french-lab-publishes-benchmark-for-long-horizon-robot-agents?lang=es","language":"es","url":"https://newsverum.com/es/news/french-lab-publishes-benchmark-for-long-horizon-robot-agents"},{"api_url":"https://newsverum.com/api/v1/articles/french-lab-publishes-benchmark-for-long-horizon-robot-agents?lang=de","language":"de","url":"https://newsverum.com/de/news/french-lab-publishes-benchmark-for-long-horizon-robot-agents"},{"api_url":"https://newsverum.com/api/v1/articles/french-lab-publishes-benchmark-for-long-horizon-robot-agents?lang=pt","language":"pt","url":"https://newsverum.com/pt/news/french-lab-publishes-benchmark-for-long-horizon-robot-agents"}],"url":"https://newsverum.com/en/news/french-lab-publishes-benchmark-for-long-horizon-robot-agents"},"version":"1"}