Hyunjong Ok and Jaeho Lee, Pohang University of Science and Technology · Weight-space architecture
Weight-Space Programming Doesn't Need Transformers
The paper studies whether LLMs writing neural network weights by hand, without gradient training, should still use Transformer architectures. Three frontier LLMs, given ten labelled examples and a sandbox, mostly on multiple-choice tasks, compile plain-text facts into a sparse table when free to design the architecture, reaching up to 54.4% test accuracy. The authors report an LLM-designed model beating the most accurate Transformer the same LLM wrote, while using about 1,800x fewer parameters, and propose an associative-memory model up to 17x faster on GPUs than a same-sized Transformer. Pretrained LLMs remain more accurate on the tasks tested.
Open the source ↗Similar Research & Safety releases
- Multi-Branch Transformer
Gaotang Li, Lingjie Chen, Chunyuan Deng, Zhiwei Xu, Hongzheng Chen, Christopher De Sa, Jingrui He, Yingheng Wang · Branching Transformer · Week 48, 2026
- DySENSE
Kailai Sun, Zijie Gao, Mingyi He, Can Rong, Alok Prakash, Baoshen Guo, Shenhao Wang, Jinhua Zhao (Massachusetts Institute of Technology) · Urban Energy GenAI · Week 48, 2026
- AllocFlow
Annan Wang, Yuchen Xie, Yu Cen, Leon Liangyu Chen, Qianyu Julie Zhu, Xin Li, Tianyi Wei, Weisi Lin (Nanyang Technological University) · Flow-Matching Trading · Week 48, 2026
- MIRIX-2.0: Multi-Agent Memory System
Wei-Chieh Huang, Weizhi Zhang, Jiayan Nan, Peng Xia, Yu Wang (arXiv) · Agent Memory System · Week 48, 2026
- MMFlash: Faster Speculative Decoding for Vision-Language Models
Sicheng Feng, Zigeng Chen, Song Wang, Gongfan Fang, Xinchao Wang (National University of Singapore) · Multimodal Speculative Decoding · Week 48, 2026
- A step toward defining the Standard Model on a lattice
OpenAI · AI-Assisted Proof · Week 48, 2026
Published by Quanta AI Labs, an AI agency in India. Browse every week in AI News Weekly.