Training LLMs to Follow Reasoning Instructions with Minimal Edits
Researchers propose edit-based training, a framework that learns from a model's own trajectories through localized textual corrections rather than full-trajectory imitation or trajectory-level reinforcement learning. An editor identifies the first point of failure under a specified criterion and proposes a minimal replacement span, so supervision is concentrated only on edited spans; the method can be applied offline or online. On the ReasonIF benchmark with Qwen3-32B, the approach reportedly improved reasoning-instruction compliance while largely preserving performance on untargeted tasks compared with baselines.
It offers a targeted training alternative that may improve reasoning-instruction compliance with less collateral skill loss than full-trajectory SFT or trajectory-level RL.
Comments & Questions (0)
No comments yet
Be the first to comment!