Research
StepBack: Step-Level Error-Localized Resampling for Efficient Test-Time Reasoning
StepBack is a new decoding strategy that improves test-time reasoning efficiency by localizing likely first errors in draft reasoning trajectories and resampling only the error step and its suffix, followed by majority voting. It matches the accuracy of standard self-consistency across seven math and multi-hop QA benchmarks while reducing token usage by up to 41.26%. The method can also be combined with adaptive inference methods like Adaptive Consistency for further efficiency gains.
efficient inferenceLLM reasoningself-consistency