UC Berkeley, Impossible and UT Austin · Video Delta Attention

VDN

Video DeltaNet is a hybrid attention design pairing sliding-window softmax attention with a linear-attention branch, aimed at the more than 85% of runtime softmax attention eats in video diffusion, without the identity, layout and temporal-consistency damage linear attention usually brings. Applied to MiniMax H3 the authors report 2.65x per layer - 125.3ms against 332.5ms on one B200 - a 14.3-second 768p clip in 11.23 seconds on eight B200s, and up to 74.5x against a dense single-GPU baseline. Code and weights released.