Research Editor selected
Artifact-to-Capability: Diagnosing Breaks in Capability Formation of Self-Evolving Agents
This paper introduces a formal definition of capability formation from artifacts in self-evolving agents, proposing five sequentially dependent properties and a three-scale diagnosis method. In freestyle Gomoku, Reflexion, Voyager, and DGM all show standard proxy signals but exhibit distinct capability-formation breaks, such as legalization gain, compositional override, and loss-only drift. The findings demonstrate that signal-based evaluation cannot reliably determine whether a capability has formed.
Why it matters: Provides a concrete diagnostic framework and counterexamples that challenge common evaluation practices in self-evolving AI agents, offering practical value for researchers and developers.
self-evolving agentscapability evaluationagent diagnostics