Yanke Song, Shuo Chen, Chris Rockwell, Ming-Hsuan Yang, Xun Huang et al. · Pixel-space video diffusion

PixVid: Back to Pixels, Lifting Latent Video Diffusion into Pixel Space

PixVid finetunes a pretrained latent video model, Wan2.1-14B, to generate every frame directly in pixels. It keeps the diffusion transformer, replaces the VAE with a linear patch embedding and a causal convolutional detailer head, and finds that predicting clean video with a much larger noise shift is critical to quality. The authors report quality competitive with the original latent model within 20K iterations. Working frame by frame, it generates 1664 × 960 video with the same 30 × 52 tokens per frame as the 832 × 480 model, keeps depth-conditioned editing aligned under fast motion, and applies each action on the next frame rather than waiting for a 4-frame block (up to 133 ms of delay at 30 FPS). The page does not state licence or code availability.

Open the source ↗

Similar Video & Animation releases

Published by Quanta AI Labs, an AI agency in India. Browse every week in AI News Weekly.