Back to AI News
arxiv

Agent-Conditioned Routing Elicits More Distinct Methods From One Frozen MoE Model

Agent-Conditioned Routing (ACR) runs parallel trajectories inside one frozen sparse mixture-of-experts model by applying persistent, untrained shared and private expert-eligibility masks while keeping the learned, input-dependent token router. On a screened 126-question mathematics and physics benchmark, ACR yielded about twice as many problems with at least two distinct primary methods among soft-correct solutions as the strongest temperature setting on GLM 5.3 (37.3 versus 18.7) and 1.67 times as many on GPT-5.6 Sol (45 versus 27), at matched output budgets. A blinded four-expert audit of 30 problems also found more multi-method solutions with ACR on both backbones, though correctness was lower. ACR adds about 160 KiB of mask state, and a three-problem microbenchmark stayed within 3% of batched temperature sampling on latency and peak VRAM.

AI Brief

It reports a concrete, low-overhead routing method that increases distinct solution methods from a single frozen MoE model without fine-tuning or extra copies.

arXiv:2610.37375v1 Announce Type: new Abstract: Generating several solutions can expose alternative methods, but additional trajectories often paraphrase one dominant approach. We investigate whether the conditional capacity of a frozen sparse mixture-of-experts (MoE) model can provide such diversity from one problem prompt, without user-assigned roles, prescribed methods, fine-tuning, or model copies. Agent-Conditioned Routing (ACR) assigns $m$ parallel trajectories persistent, untrained shared/private expert-eligibility masks while preserving the learned, input-dependent token router. On a screened 126-question multi-mode mathematics and physics benchmark, with model judges reading the reported derivations in full, we count the number of problems yielding at least two distinct primary methods among soft-correct solutions in four trajectories. ACR reaches approximately twice as many such problems as the strongest temperature setting under GLM 5.3 (37.3 versus 18.7), and 1.67 times as many under GPT-5.6 Sol (45.00 versus 27.00), at matched output budgets. A blinded four-expert audit on a stratified sample of 30 problems also finds more multi-method solutions with ACR on both backbones, with lower correctness. Matched-eligibility ablations show modest multi-method count advantages for ACR, while token-resampled masks attain higher correctness. An informed sequential prompt elicits more diversity still, using previous solutions and explicit requests for alternative methods. ACR adds about 160 KiB of mask state. In a three-problem microbenchmark capped at 1,024 tokens, its latency and peak VRAM usage remain within 3% of batched temperature sampling.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!