AstroMemBench: A Benchmark for Memory Systems in Embodied Spacecraft Agents
Researchers introduce AstroMemBench, a benchmark for evaluating long-term memory systems in embodied spacecraft agents operating under communication delays, resource limits and safety constraints. The benchmark uses a Scenario-Challenge-Capability design method and covers 16 memory subchallenges through 140 templates and 2240 instances across nine tasks in planetary surface operations, on-orbit servicing and deep-space exploration. A deterministic environment scores actions for task completion, state changes, safety and resource use, and ablations under nine conditions test sensitivity to memory recovery and design differences. Experiments comparing six configurations find that Mem0, Chain-of-Memory, LightMem and RAG improve same-session mean scores over a native Pi Agent, with no clear advantage of the three memory systems over RAG, while Mem0 outperforms RAG by 12.25 points after a session restart. The benchmark and evaluation code are publicly released.
It provides a concrete, openly available benchmark and comparative evidence for memory-system design in embodied spacecraft agents, useful to researchers working on long-horizon embodied AI.
Comments & Questions (0)
No comments yet
Be the first to comment!