Agentic Learning AI Lab (Jack Lu, Zhenbang Yang, Mike Lasby, Tejas Pote, Yani Ioannou, Mengye Ren) · Weight Deletion Tuning
The Surprising Effectiveness of Deleting Weights in LLM Post-Training
The paper introduces Deletion Tuning, which zeroes fewer than 0.05% of an LLM's weights by learning a binary mask over the gate projection of transformer blocks, with no other modification. The authors report that it matches full fine-tuning across reinforcement learning with verifiable rewards, supervised fine-tuning and on-policy distillation. The trained mask is hundreds to thousands of times smaller than the corresponding full fine-tuning checkpoint, and under reinforcement learning its prior-task accuracy stays near base level while full fine-tuning, LoRA and random sparse fine-tuning lose up to 5.4 points.
Open the source ↗Similar Research & Safety releases
- Human-in-the-loop Topology Optimisation Copilot
MIT (Dat Ha, Josephine Carstensen, Ferdous Alam, Markus Buehler, Faez Ahmed) · Design Copilot · Week 47, 2026
- Read Less, Solve More: Token-Efficient Sparse Reading of External Content for AI Agents
Zedong Liu, Jiaan Wu, Xinyang Ma, Le Xu, Kai Wang, Yuanchao Hu, Dingwen Tao, Guangming Tan (Institute of Computing Technology) · Token-efficient agent reading · Week 47, 2026
- On the Design of Hyper-Connections: Simplification, Stability, and Scalability
Xiangdong Zhang et al. (dots studio) · Simpler Hyper-Connections · Week 47, 2026
- SteerDistill: Quantizing LLM Agents via Closed-Loop Teacher Steering
Genming Cui, Xiangyu Li, Zixu Hao, Xin Ding, Yunxin Liu, Ting Cao · Agent Quantisation Distillation · Week 47, 2026
- Theoretical Limits of Language Model Alignment
Apple Machine Learning Research (Lucas Monteiro Paes, Natalie Mackraz, Barry-John Theobald, Federico Danieli) · Alignment limits theory · Week 47, 2026
- Incentivizing Self-Learning in Large Language Models via Reinforcement Learning
Lingkai Kong, Yuzhe Gu, Haiteng Zhao, Zijian Wu, Youqing Fang, Zhouqi Hua, Yanan Sun, Kai Chen (Intern Large Models) · RL Self-Learning · Week 47, 2026
Published by Quanta AI Labs, an AI agency in India. Browse every week in AI News Weekly.