Back to AI News
arxiv

Interpretable weight-space programming builds transparent transformers and brain encoding models

The paper introduces interpretable weight-space programming (iWSP), a method where an LLM agent constructs neural network weights and features directly in code without backpropagation, enabling human-interpretable models. iWSP successfully builds compact transformers for arithmetic and sentiment tasks)Skip, and outperforms interpretable baselines in brain encoding models across vision, language, and audition, even uncovering data confounds and interpretable acoustic features. This approach aligns predictive performance with scientific explanation.

AI Brief

This is a novel method that combines LLM-driven model construction with interpretability, showing concrete results across multiple domains, offering high value for AI and neuroscience researchers.

arXiv:2611.18114v1 Announce Type: cross Abstract: Neural networks are powerful predictors, yet their opacity limits their usefulness in science, where the goal is explanation rather than mere prediction. In neuroscience, for example, deep encoding models accurately predict brain activity in response to natural stimuli, but struggle to reveal which stimulus features drive those predictions. Here we introduce interpretable weight-space programming (iWSP), a method in which an LLM agent iteratively builds interpretable models by explicitly programming their weights and features in code, without gradient-based training (backpropagation). Without backpropagation to tune parameters, the agent must draw on its prior knowledge and reason over previous trials to hypothesize, hand-craft, and refine human-interpretable computations mirroring the workflow of a human scientist. LLM audits confirm the interpretability of the hand-crafted models. We first validate iWSP by tasking it with constructing transformers for canonical benchmarks such as arithmetic and sentiment classification, finding that LLM agents can successfully craft compact, accurate networks directly. Next, we apply iWSP to build neural encoding models across vision, language, and audition. In the visual cortex, iWSP outpredicts existing interpretable baselines, uncovers distinct computations across early/higher visual areas, and generalizes to unseen data. In language fMRI, iWSP surfaces a confound in the data and in auditory ECoG, it systematically discovers interpretable acoustic features that predict neural activity. Together, these results establish iWSP as a general paradigm for aligning high predictive performance with human-interpretable scientific discovery.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!