RikiyaT (checkpoint built by akhaliq with ML Intern) · LLM-edited weights

Qwen3.5-0.8B-weight-space-programming

This is Qwen3.5-0.8B with its weights edited directly by a frontier LLM, called "Astra", with no gradient descent or training. It is Experiment 8 from the post "Sparks of Weight-Space Programming". Astra worked in a Python session with a budget of 50 forward passes on practice problems from the GSM8K and MATH training sets. It traced the base model's thinking loops to one direction written by the MLPs of layers 15 and 19, then subtracted it from 64 output columns of `down_proj` in each layer (131,072 weights, 0.016% of the model). In the post's evaluation (thinking on, up to 32,768 tokens), MATH-500 rose from 7.9% to 48.1%, GSM8K from 13.8% to 67.1% and IFEval from 21.0% to 62.3%. The share of answers hitting the token limit fell from 93.0% to 33.1%. An independent re-run of the full MATH-500 (2,000 generations) gave 51.0%, with 31.6% hitting the limit. Only two tensors differ from the base model, and the edit is published as a bit-exact diff with sha256 verification. The checkpoint is released under Apache 2.0, and a live demo is available on Hugging Face Spaces.