Beihang University, VAST, Tsinghua University, University of Hong Kong · 3D Generation

MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generation

MIDI extends pre-trained image-to-3D models into multi-instance diffusion models that simultaneously generate multiple 3D objects with accurate spatial relationships from a single image using novel multi-instance attention mechanisms. The method processes complete scenes in ~40 seconds by conditioning on segmented object images and global scene context.