Back to AI News
arxiv

Proximal Targets: Training Looped Refiners on Problems with Many Correct Answers

Researchers identify a loss floor in looped refiners such as HRM and TRM when deep supervision trains every pass toward a single stored answer for puzzles with multiple valid solutions, and show the loss-minimising pass can blend answers into an invalid output. They propose proximal targets, which train each pass toward the correct answer nearest its own input, found via an answer list, a search solver started at the draft, or a quality band. With a 1.7M-parameter network and 5 seeds, the method solves 98.8% of Sudoku puzzles with 2-64 answers versus 2.5% for the standard target, and 93.8% of sparse puzzles with more than 10,000 answers without an answer list versus 0.0%. It also reaches 90.9% on 200-node 3-colourings after training on graphs of at most 50 nodes, compared with 36.9% for the baseline.

AI Brief

It offers a concrete training fix for looped reasoning models, with substantial reported gains on multi-solution puzzles that could inform work on iterative refinement.

arXiv:2611.22737v1 Announce Type: new Abstract: Looped refiners such as HRM and TRM apply one small network repeatedly to a draft answer, and deep supervision trains every pass toward one stored answer per puzzle. We prove that when a puzzle has many correct answers this target sets a loss floor that no model capacity removes, and that the loss-minimising pass blends the answers into an invalid grid. We propose the proximal target: train each pass toward the correct answer nearest to its own input, found by scanning a list of answers, by a search solver started at the draft, or within a band of quality set by a knob; correct answers become fixed points, and a pass only has to make fewer errors than its input has for the loop to converge, which its training loss bounds. With the same 1.7M-parameter network and 5 seeds, the proximal target solves 98.8% of Sudoku puzzles with 2-64 answers (standard target: 2.5%), 93.8% of sparse puzzles with more than 10,000 answers without any list of answers (0.0%), and 90.9% of 200-node 3-colourings after training on graphs of at most 50 nodes (36.9%), and different random starts end at different answers. Against GRAM, a stochastic-latent reasoner, a single run is correct far more often (98.8% vs. 57.7%) and twenty runs find more of the answers (53.6% vs. 37.4%), while GRAM is slightly ahead on its own 10-node colouring benchmark; in Scrabble the knob becomes a calibrated strength dial.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!