Zeguan Xiao, Zhuocheng Liang, Yun Chen, Yong Wang, Guanhua Chen, Pinyan Lu · Curvature-Constrained Adaptation

CurvDelta: Low-Curvature Delta Modeling For Surgical Learning and Unlearning

CurvDelta is a framework that builds capability preservation into the parameterisation of LLM updates, using a low-curvature space around the pre-adaptation model, approximated with K-FAC and a factorised projection. The authors also introduce CurvLoRA, a low-rank variant for more efficient adaptation. The paper reports competitive maths reasoning and code generation with better retention of general capabilities, and a better forgetting-retention balance than baselines on TOFU and MUSE for knowledge unlearning.

Open the source ↗

Similar Research & Safety releases

Published by Quanta AI Labs, an AI agency in India. Browse every week in AI News Weekly.