Amazon (Alex M Tseng, Luca Zancato), via Hugging Face blog · MoE Expert Pruning

HOPE: Higher-Order Pruning of Experts

HOPE is a second-order method for pruning experts in Mixture-of-Experts language models. Unlike first-order methods that score each expert independently, it accounts for pairwise interactions between co-activated experts when choosing which to remove. The authors report that at 50% pruning HOPE achieves the best average rank of any method tested, with gains of up to +6.1% on agentic coding tasks over the next-best pruning method. The post links to a paper with the full derivation; licence and code availability are not stated on the page.