NVIDIA and University of Waterloo · Encoder-Free Unified Model

PixelUMM

A unified model that both understands and generates images and video from raw pixels with one decoder-only Transformer and no VAE or separate vision encoder. Images enter as 16x16 patches and video as four-frame tubelets, and text, clean pixels and noisy pixels share self-attention on a Qwen3-8B backbone with separate understanding and generation experts. It covers image and video understanding and generation, image-to-video and video editing, in 1.7B and 8B sizes. NVIDIA reports DocVQA 90.42, ChartQA 82.96, AI2D 80.12, MVBench 70.53 and Video-MME 57.33 for the 8B model, vendor-reported. Paper 29 September (arXiv 2609.38597), weights 1 October; the code says Apache-2.0 while the weights are tagged "other".