Live intelligence

Latest AI updates

7 updates
Research

WeightBench: Benchmarking LLMs as Weight-Space Programmers

The paper introduces WeightBench, a benchmark for evaluating how well large language models can read, modify, and create neural-network weights without gradient descent. It covers 42 tasks across five tracks, including weight reading, fixing, writing, recognizing impossible edits, and data-free compression, on models ranging from 2K-parameter policies to 7B language models. Results show that frontier agents can handle some small-network tasks but still struggle to patch language models, compress them without data, or reliably identify impossible edits.

Large language modelsWeight-Space ProgrammingAI benchmarks
Products

Cmpnd Releases dspy-wsp, a DSPy Optimizer That Edits Model Weights Without Gradients

Cmpnd has released dspy-wsp, an MIT-licensed prototype that packages weight-space programming as a DSPy optimizer called WeightSpace, which plugs into existing DSPy programs and metrics. A frontier writer LLM reads a small student model's weight tensors inside a sandbox and writes targeted patches, working within a fixed forward-pass budget (25 passes in the reported tests) with no gradients or training loop. On Cmpnd's Typed Decisions benchmark, runs costing roughly $5.42 to $7.63 improved decision models from 0.6B to 9B parameters, with d1-omni-600M rising from 50.9% to 69.1% when WeightSpace was combined with the earlier ReAnchor optimizer. Cmpnd reports the approach is experimental, slow (78 to 167 minutes per run) and weakest on the largest model tested, where WeightSpace alone added only 1.9 points.

DSPyWeight-space programmingCmpnd
Research

Weight-Space Programming Compiles Known Game Rules into Compact Neural Agents

This paper tests whether language models can compile known game mechanics directly into neural network weights, without training, using 20 unpublished ARC-AGI-3-style game families. With a fixed 57k-parameter recurrent architecture, the strongest writer reaches 56% held-out-level success with feedback, while Python players reach 97-100%. Written networks are highly sparse and structured, but remain fragile and do not combine successfully with PPO fine-tuning.

weight-space programmingARC-AGI-3neural network compilation
Research

Mixed Signals: Weight-Space Programming for EEG and ECG

This arXiv paper applies weight-space programming, in which language models read and edit trained neural-network weights, to biosignal models trained on EEG and ECG data. It reports that frontier models could debug planted faults in such models and write network weights from scratch given the architecture, task and a minimal set of training subjects. In one experiment, a language model matched standard gradient-descent training using 6x fewer training subjects; networks written by Claude Opus 5.5 also outperformed classic hand-crafted EEG pipelines built from the same five subjects. The authors note limitations, including an inability to improve already well-trained classifiers or reliably infer the training task from weights alone, and conclude the approach is promising for small edge and wearable models but needs further investigation.

weight-space programmingEEG/ECG modelslanguage models
Research

Zero-Shot Weight-Space Programming Across a Difficulty-Graded Ladder of Two-Player Games

This arXiv paper tests how far a large language model can perform weight-space programming zero-shot, setting every weight of a feed-forward network by hand from an explicit specification without gradient descent or training data. The authors design a four-rung ladder covering Nim, tic-tac-toe, two sizes of Connect Four, and a chess endgame, and build a second independent network at each rung using a structurally unrelated game at matched difficulty. Ten networks were built and independently verified, with six admitting a complete non-statistical proof of correctness against an independent oracle. The chess-endgame network is exact for move legality and one-ply mate detection but collapses on multi-move mating plans, dropping from 84.1-100% success on one-ply tactics to 0.57-9.13% in full adversarial play; the authors report that holding hand-set weights fixed and adding shallow search confirms an architectural ceiling, and they release all networks, oracles, construction-cost measurements, and a cold-prompt protocol.

weight-space programmingLLM reasoninggame-playing benchmarks
Research

Weight-Space Programming Shows Localized Gains but Uneven Generalization

A systematic study evaluates Weight-Space Programming across 16 editing trajectories, seven benchmarks, and two Qwen3.5 model scales. It reports accuracy gains of up to 29.45 points on MATH-500, 4.42 on GPQA Diamond, 36.72 on IFEval, and 17.11 on SciBench, with a 29.45-point MATH-500 improvement from modifying only 1,023 parameters. Cross-task transfer was promising but uneven: a Math-edited checkpoint improved GPQA Diamond, IFEval, and SciBench, while SciCode performance was unchanged. Editing reliability remained inconsistent, and six of 16 trajectories retained the original weights when candidate improvements could not be validated.

Weight-Space ProgrammingModel editingCross-task generalization
Research Editor selected

Weight-Space Programming: Frontier Models Directly Edit Neural Network Weights

Multiple groups report that frontier LLMs can generate or edit small networks' weights directly, without gradient training or tools. Replications include Alex Krentsel's reproduction of MIT's experiment, Hugging Face's reproduction with ML Intern, and Simon Willison's laptop run with Qwen 3.8 27B. Skeptics note missing baselines and potential benchmark overfitting, while supporters see meta-learning finally working.

Why it matters: This is a concrete, recent AI development with multiple independent replications, sparking debate about model capabilities and interpretability.

weight-space programmingfrontier modelsneural network editing