Back to AI News
arxiv

GeoMind benchmark targets spatial reasoning and memory in robotic manipulation

Researchers introduced GeoMind, a benchmark for structured spatial understanding in tabletop robotic manipulation. It contains 54 evaluated task variants focused on spatial reasoning and spatial memory, along with a scalable pipeline for generating and annotating task instances with verified targets and executable interactions. Evaluations across embodied agents found consistent weaknesses, including difficulty tracking identities after spatial changes and translating relational or geometric constraints into actions. The benchmark is released publicly on GitHub.

AI Brief

It gives embodied-AI researchers a concrete new benchmark specifically isolating spatial reasoning and spatial memory weaknesses in manipulation agents.

arXiv:2611.12175v1 Announce Type: cross Abstract: Robotic manipulation has advanced rapidly in mapping visual observations and language instructions to actions. However, reliable manipulation in open-world environments requires robust spatial understanding: the ability to identify targets in complex layouts (spatial reasoning) and adapt to scenes that continuously evolve through interaction (spatial memory). Yet existing embodied benchmarks fall short in evaluating spatial understanding: they often introduce strong visual cues that bypass spatial reasoning, and focus on static scenes that offer little insight into an agent's capacity for spatial memory. To address this gap, we introduce GeoMind, a systematic benchmark for structured spatial understanding through tabletop manipulation. GeoMind comprises 54 evaluated task variants organized around spatial reasoning and spatial memory. It also incorporates a scalable pipeline that automates the generation and annotation of task instances with verified targets and executable interactions. Extensive evaluations across diverse embodied agents reveal their consistent weaknesses in spatial understanding. Our findings highlight that these agents struggle to track identities after spatial changes and convert relational or geometric constraints into executable actions. We release the GeoMind benchmark at https://github.com/Notfound-JH/GEOMIND-benchmark.git.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!