Back to AI News
arxiv

ARCHER: From Rationale Imitation to Executable Reasoning in Small Language Models

ARCHER is a two-stage framework for training small language models to produce executable, auditable reasoning trajectories for mathematical tasks. The first stage compiles free-form chain-of-thought rationales into typed action traces with reasoning steps, calculator calls, runtime observations, verification actions, and answers. The second stage refines the policy using asymmetric hierarchical reinforcement learning with grouped and conditionally gated trajectory rewards. Across five sub-10B backbones, ARCHER reportedly improves macro-average accuracy by 5.51 percentage points over the strongest tool-augmented baseline, including gains on MATH-500, AIME-2024, and AIME-2025.

AI Brief

It presents a concrete training method with reported benchmark gains for making small language models more reliable at tool-grounded mathematical reasoning.

arXiv:2611.18494v1 Announce Type: new Abstract: Small language models are attractive for local, low-cost, and privacy-sensitive deployment, but they remain unreliable on mathematical reasoning tasks that require decomposition, exact computation, tool use, and verification. We introduce \textbf{ARCHER}: \textbf{A}ction-trace \textbf{R}einforcement for \textbf{C}alculator-grounded \textbf{H}ierarchical \textbf{E}xecutable \textbf{R}easoning, a two-stage framework for training small language models (SLMs) to produce executable and auditable reasoning trajectories. In the first stage, ARCHER compiles free-form chain-of-thought rationales into typed action traces containing reasoning steps, calculator calls, deterministic runtime observations, verification actions, and final answers. This shifts the supervision target from imitating explanatory text to learning a tool-grounded reasoning policy. In the second stage, ARCHER refines this policy through asymmetric hierarchical reinforcement learning, using grouped and conditionally gated trajectory rewards for answer correctness, tool grounding, process discipline, and trace alignment. Across five standard sub-10B backbones, ARCHER improves over the strongest tool-augmented baseline by 5.51 percentage points on macro-average across benchmark accuracies, including gains of 8.95 points on MATH-500 and 5.33 and 5.34 points on AIME-2024 and AIME-2025. Evaluations on two additional reasoning-specialized 7B backbones further show that executable grounding remains beneficial after reasoning-focused reinforcement learning or distillation. These results demonstrate that executable supervision and reward-guided behavioral refinement provide a general and effective training paradigm for compact mathematical reasoners.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!