Hyunjong Ok and Jaeho Lee, Pohang University of Science and Technology · Weight-space architecture

Weight-Space Programming Doesn't Need Transformers

The paper studies whether LLMs writing neural network weights by hand, without gradient training, should still use Transformer architectures. Three frontier LLMs, given ten labelled examples and a sandbox, mostly on multiple-choice tasks, compile plain-text facts into a sparse table when free to design the architecture, reaching up to 54.4% test accuracy. The authors report an LLM-designed model beating the most accurate Transformer the same LLM wrote, while using about 1,800x fewer parameters, and propose an associative-memory model up to 17x faster on GPUs than a same-sized Transformer. Pretrained LLMs remain more accurate on the tasks tested.

Open the source ↗

Similar Research & Safety releases

  • Multi-Branch Transformer

    Gaotang Li, Lingjie Chen, Chunyuan Deng, Zhiwei Xu, Hongzheng Chen, Christopher De Sa, Jingrui He, Yingheng Wang · Branching Transformer · Week 48, 2026

  • DySENSE

    Kailai Sun, Zijie Gao, Mingyi He, Can Rong, Alok Prakash, Baoshen Guo, Shenhao Wang, Jinhua Zhao (Massachusetts Institute of Technology) · Urban Energy GenAI · Week 48, 2026

  • AllocFlow

    Annan Wang, Yuchen Xie, Yu Cen, Leon Liangyu Chen, Qianyu Julie Zhu, Xin Li, Tianyi Wei, Weisi Lin (Nanyang Technological University) · Flow-Matching Trading · Week 48, 2026

  • MIRIX-2.0: Multi-Agent Memory System

    Wei-Chieh Huang, Weizhi Zhang, Jiayan Nan, Peng Xia, Yu Wang (arXiv) · Agent Memory System · Week 48, 2026

  • MMFlash: Faster Speculative Decoding for Vision-Language Models

    Sicheng Feng, Zigeng Chen, Song Wang, Gongfan Fang, Xinchao Wang (National University of Singapore) · Multimodal Speculative Decoding · Week 48, 2026

  • A step toward defining the Standard Model on a lattice

    OpenAI · AI-Assisted Proof · Week 48, 2026

Published by Quanta AI Labs, an AI agency in India. Browse every week in AI News Weekly.