Back to AI News
arxiv

Weight-Space Programming Shows Localized Gains but Uneven Generalization

A systematic study evaluates Weight-Space Programming across 16 editing trajectories, seven benchmarks, and two Qwen3.5 model scales. It reports accuracy gains of up to 29.45 points on MATH-500, 4.42 on GPQA Diamond, 36.72 on IFEval, and 17.11 on SciBench, with a 29.45-point MATH-500 improvement from modifying only 1,023 parameters. Cross-task transfer was promising but uneven: a Math-edited checkpoint improved GPQA Diamond, IFEval, and SciBench, while SciCode performance was unchanged. Editing reliability remained inconsistent, and six of 16 trajectories retained the original weights when candidate improvements could not be validated.

AI Brief

It provides researchers with concrete evidence on the performance gains, cross-task transfer, and reliability limits of direct weight-space model adaptation.

arXiv:2611.09382v1 Announce Type: new Abstract: Weight-Space Programming (WSP) enables reasoning agents to directly inspect and modify neural network parameters, offering an alternative to gradient-based model adaptation. However, its capabilities, cross-task generalization, and practical limitations remain insufficiently understood. We systematically investigate WSP through 16 editing trajectories across seven benchmarks and two model scales (Qwen3.5-0.8B and Qwen3.5-9B). Our findings reveal three key insights. First, WSP can achieve substantial performance gains through highly localized interventions. On full held-out evaluations, accuracy improves by up to 29.45, 4.42, 36.72, and 17.11 percentage points on MATH-500, GPQA Diamond, IFEval, and SciBench, respectively. Notably, modifying only 1,023 parameters yields a 29.45-point improvement. Second, WSP exhibits promising but uneven cross-task generalization. A Math-edited checkpoint improves GPQA Diamond, IFEval, and SciBench by 3.03, 12.18, and 20.10 points, respectively, without further editing, while leaving SciCode performance unchanged. Third, effective editing remains challenging and inconsistent. Outcomes vary substantially across trajectories, and six of the 16 trajectories retain the original weights when candidate improvements cannot be reliably validated. Together, these results highlight WSP's potential for highly localized model adaptation while revealing important challenges in intervention discovery, reliability, and cross-task transfer. Homepage: https://wsp-scope.github.io/ GitHub: https://github.com/WSP-Scope/WSP-Scope-Project
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!