arXiv:2611.12175v1 Announce Type: cross
Abstract: Robotic manipulation has advanced rapidly in mapping visual observations and language instructions to actions. However, reliable manipulation in open-world environments requires robust spatial understanding: the ability to identify targets in complex layouts (spatial reasoning) and adapt to scenes that continuously evolve through interaction (spatial memory). Yet existing embodied benchmarks fall short in evaluating spatial understanding: they often introduce strong visual cues that bypass spatial reasoning, and focus on static scenes that offer little insight into an agent's capacity for spatial memory. To address this gap, we introduce GeoMind, a systematic benchmark for structured spatial understanding through tabletop manipulation. GeoMind comprises 54 evaluated task variants organized around spatial reasoning and spatial memory. It also incorporates a scalable pipeline that automates the generation and annotation of task instances with verified targets and executable interactions. Extensive evaluations across diverse embodied agents reveal their consistent weaknesses in spatial understanding. Our findings highlight that these agents struggle to track identities after spatial changes and convert relational or geometric constraints into executable actions. We release the GeoMind benchmark at https://github.com/Notfound-JH/GEOMIND-benchmark.git.
arxiv
GeoMind benchmark targets spatial reasoning and memory in robotic manipulation
Researchers introduced GeoMind, a benchmark for structured spatial understanding in tabletop robotic manipulation. It contains 54 evaluated task variants focused on spatial reasoning and spatial memory, along with a scalable pipeline for generating and annotating task instances with verified targets and executable interactions. Evaluations across embodied agents found consistent weaknesses, including difficulty tracking identities after spatial changes and translating relational or geometric constraints into actions. The benchmark is released publicly on GitHub.
It gives embodied-AI researchers a concrete new benchmark specifically isolating spatial reasoning and spatial memory weaknesses in manipulation agents.
Comments & Questions (0)
No comments yet
Be the first to comment!