Google Research (Ali Behrouz, Vahab Mirrokni) · Sleep-phase continual learning

Sleep framework for continual learning

Google Research introduces a sleep framework for continual learning in large language models, aimed at the static knowledge cutoff and catastrophic forgetting that affect deployed models. The model alternates between an active (wake) phase, where new input is held in short-term memory, and an offline sleep phase. During sleep it unlocks pre-allocated Mixture-of-Experts memory blocks and updates its weights in two stages. In memory consolidation, short-term context is distilled into new low-rank long-term experts using on-policy distillation and a reward based on Levenshtein similarity, after which the temporary parameters are reset. In dreaming, the model generates synthetic samples using randomly selected experts, filters them by gradient-based selection and refines itself with RL and supervised fine-tuning. The authors report near-zero performance drop on continual translation in unseen languages (Manchu and Kalamang), where in-context learning, Cartridges and SFT forgot earlier material, and near-perfect scaling up to 10M tokens on BABILong, ahead of baselines including GPT-4 and Titans. The post also gives a theoretical analysis; no code release is mentioned.