Joi-Bench-Task benchmarks task coordination in full-duplex voice agents
The paper introduces Joi-Bench-Task, a benchmark designed to isolate coordination between task execution and conversation in full-duplex voice agents. It uses an interactive user simulator that injects configurable interventions during tool execution and spoken result delivery for reproducible evaluation. Five commercial voice agents are assessed on execution responsiveness, coordination under request changes, and coordination under interruptions. The reported results show a maximum 55% result completion rate under request changes and average information omission rates of at least 23% after interruption for four of the five agents.
It provides a reproducible benchmark and empirical evidence on task-coordination failures that matter for builders and evaluators of voice agents.
Comments & Questions (0)
No comments yet
Be the first to comment!