Back to AI News
arxiv

The Value of Thinking: Learning When to Reason in Hybrid Decision Models

The paper proposes a zero-or-T thinking paradigm in which a single model either answers from implicit computation in one forward pass without generating tokens or performs full autoregressive reasoning when the expected benefit justifies the token cost. It implements this as a hybrid decision model with a frozen backbone and a lightweight self-routing block that estimates the value of thinking against a per-request price. For settings without per-decision labels, such as episodic agentic tasks, the authors introduce Calibrated Decision Policy Optimisation to train the routing block from episode-level returns. Evaluations on MMLU-Pro, AGIEval, GPQA-Diamond, JevBench, BANKING77 and MiniWoB report accuracy and token-cost improvements, including a 72.5% median latency reduction on JevBench and an 80.1% token-cost cut on MiniWoB with HDM-27B.

AI Brief

It presents a concrete routing and reinforcement-learning method for reducing reasoning token costs while preserving accuracy across QA and agentic tasks.

arXiv:2611.16961v1 Announce Type: new Abstract: Existing adaptive-reasoning methods shorten the explicit reasoning a model writes, or skip it by switching modes; either way the answer is still written out token by token. \emph{Rather than deciding how much to think}, we explore a different paradigm, \emph{zero-or-$T$ thinking}: a single model either answers from the implicit thinking inside one forward pass, generating no token, or performs full autoregressive reasoning at a cost of $T$ tokens when that is worth paying. We realise it in a \emph{hybrid decision model} (HDM), where one frozen backbone serves both modes and a lightweight \emph{self-routing block} chooses between them. The block estimates the \emph{value of thinking}, the utility the slow mode is expected to add over the fast one, from what that implicit thinking has already produced, and the slow mode runs only where that value exceeds a price $\lambda$ set per request. Where no per-decision label exists, as in episodic agentic tasks, we introduce \emph{Calibrated Decision Policy Optimisation} (CDPO), a reinforcement-learning recipe that trains the block from episode-level returns. We evaluate HDM on MMLU-Pro, AGIEval, GPQA-Diamond, JevBench, BANKING77 and MiniWoB. Specifically, on JevBench HDM-4B scores $0.857$ against Jev's $0.851$ ($+0.7\%$) at a median $0.179$\,s against $0.652$ ($-72.5\%$). On the agentic venue MiniWoB HDM-27B cuts token cost by $80.1\%$.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!