Dennis Menn, Tianpei Gu, Guande He, Aryan V S, Yanke Song, Nan Liu, Desai Xie, Xun Huang, Chenfeng Xu and Diana Marculescu · Mixed-resolution video diffusion

MPEG-Diffusion: Mixed-Resolution Video Generation

MPEG-Diffusion generates video with a high-resolution region and a low-resolution background, so the diffusion transformer processes fewer tokens. A linear layer on the model's intermediate features selects the high-resolution region during denoising, so no region input is needed. It also uses separate high- and low-resolution K/V caches and a decoder that merges both latents into one before decoding. On Wan2.1 14B (720p, 81 frames) with 30% of tokens it gives a 4.8× speed-up, against 5.2× for Phase-Aligned RoPE and Foveated Diffusion, which both need a region input. Its MANIQA is 0.397 against 0.293 and 0.300 for those methods, and its VBench quality score is 83.9 against 85.3 for full resolution. The page also shows an example with FLUX.2 4B for text-to-image. The paper is listed as under review, and the page does not state a licence or code release.

Open the source ↗

Similar Video & Animation releases

Published by Quanta AI Labs, an AI agency in India. Browse every week in AI News Weekly.