Xiangdong Zhang et al. (dots studio) · Simpler Hyper-Connections
On the Design of Hyper-Connections: Simplification, Stability, and Scalability
This paper studies Hyper-Connections, an architecture increasingly used in frontier language models, and identifies simpler designs. It introduces SimpleHC, which removes residual mixing and shares memory access, adding only 2.4% end-to-end training time over a vanilla residual baseline at the largest tested scale. The authors report performance comparable to mHC when scaling from 15B to 30B MoE models, with smoother gradient dynamics.
Open the source ↗Similar Research & Safety releases
- Human-in-the-loop Topology Optimisation Copilot
MIT (Dat Ha, Josephine Carstensen, Ferdous Alam, Markus Buehler, Faez Ahmed) · Design Copilot · Week 47, 2026
- Read Less, Solve More: Token-Efficient Sparse Reading of External Content for AI Agents
Zedong Liu, Jiaan Wu, Xinyang Ma, Le Xu, Kai Wang, Yuanchao Hu, Dingwen Tao, Guangming Tan (Institute of Computing Technology) · Token-efficient agent reading · Week 47, 2026
- SteerDistill: Quantizing LLM Agents via Closed-Loop Teacher Steering
Genming Cui, Xiangyu Li, Zixu Hao, Xin Ding, Yunxin Liu, Ting Cao · Agent Quantisation Distillation · Week 47, 2026
- Theoretical Limits of Language Model Alignment
Apple Machine Learning Research (Lucas Monteiro Paes, Natalie Mackraz, Barry-John Theobald, Federico Danieli) · Alignment limits theory · Week 47, 2026
- The Surprising Effectiveness of Deleting Weights in LLM Post-Training
Agentic Learning AI Lab (Jack Lu, Zhenbang Yang, Mike Lasby, Tejas Pote, Yani Ioannou, Mengye Ren) · Weight Deletion Tuning · Week 47, 2026
- Incentivizing Self-Learning in Large Language Models via Reinforcement Learning
Lingkai Kong, Yuzhe Gu, Haiteng Zhao, Zijian Wu, Youqing Fang, Zhouqi Hua, Yanan Sun, Kai Chen (Intern Large Models) · RL Self-Learning · Week 47, 2026
Published by Quanta AI Labs, an AI agency in India. Browse every week in AI News Weekly.