Rikiya Takehi, Jayden Teoh, Alex Zhang, Omar Khattab (MIT CSAIL) · LLM weight editing

Sparks of Weight-Space Programming

MIT CSAIL researchers report early results that frontier LLMs can natively read and write neural network weights, an approach they call Weight-Space Programming. The LLM (GPT-6 Astra) works directly on raw weights from a natural-language spec, without gradient descent or training loops, and sometimes without environment feedback. Astra named the game a PPO-trained network played (four games, 17,253-weight MLPs) and identified the shape generated by small diffusion models from weights alone. Asked to improve Qwen3.5-0.8B at maths without gradients, it found a weight issue behind reasoning loops and edited 0.016% of the weights. With the same sampling settings, MATH-500 rose from 7.9% to 48.1% and IFEval rose about 3x. In another experiment it edited 3.5% of a control policy's weights in one shot, taking success from 0% to 100%. It also wrote small game-playing networks from scratch, including for MinAtar games and Go; the text states that its small game networks outplayed PPO-trained ones trained for 100 million steps. The text provided cuts off during Experiment 3 and gives no licence or code details; the post is dated 2 November 2026.