Depth as a Candidate Source: Portfolio Allocation for Recurrent Test-Time Scaling in Large Language Models
The paper analyzes recurrent latent reasoning in LLMs and finds that deeper recurrence can lose correct solutions found at shallower depths, with depth-specific success sets remaining broadly non-nested. It proposes Depth Portfolio Allocation (DPA), which treats recurrent test-time scaling as budgeted allocation across depth-specific candidate sources, supporting fixed and feedback-guided sequential allocation. Experiments on GSM8K, SVAMP, MultiArith, and MBPP indicate that mixed-depth portfolios expand oracle coverage under matched candidate counts and nominal costs, while feedback-guided allocation generally matches or improves uniform allocation and can reduce computation in high-budget settings.
It offers a concrete inference-time method and empirical analysis for researchers working on recurrent-depth LLM reasoning and test-time compute allocation.
Comments & Questions (0)
No comments yet
Be the first to comment!