Back to AI News
arxiv

Actor-Predictor Learns Formal-Specification Progress Without Automata

Researchers propose Actor-Predictor, a reinforcement learning framework that learns a surrogate representation of logic progress instead of explicitly constructing automata for formal specifications. A predictor estimates whether a partial trajectory will eventually satisfy the specification, and its learned features are supplied as auxiliary inputs to an actor optimized with GRPO and RWR. The authors show theoretically that, with sufficient predictive accuracy, the learned representation is equivalent to the automaton state. Experiments on grid-world and robotic tasks in Isaac Lab report better performance than automaton-based baselines on LTL-constrained tasks, with additional applicability to non-regular specifications such as a^n b^n.

AI Brief

It offers a concrete research method for enforcing formal temporal constraints in RL without the scalability limits of automaton construction, relevant to safe and verifiable learned control.

arXiv:2611.26313v1 Announce Type: new Abstract: One fundamental challenge in reinforcement learning (RL) is enforcing temporal safety constraints, such as satisfying formal specifications. Existing methods often transform logic formulas into automata for state augmentation and reward shaping. However, this approach assumes that automaton construction is both feasible and tractable, which limits its applicability to small specifications and linear temporal logic (LTL). This paper proposes Actor-Predictor, a framework that learns a surrogate representation of logic progress without explicitly constructing an automaton. The predictor continuously estimates whether a partial trajectory will satisfy the formal specification in a later step, and its learned features are provided as auxiliary inputs to the actor, which is then optimized using GRPO and RWR. We show theoretically that, under sufficient predictive accuracy, the learned representation is equivalent to the automaton state. Experiments on grid world and robotic tasks in Isaac Lab demonstrate that Actor-Predictor outperforms automaton-based baselines on LTL-constrained tasks. It also performs well on non-regular formal specifications, such as $(a^n b^n)$, for which finite-automaton-based approaches are not applicable.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!