NVIDIA Research (Oliver Hahn, Rodrigo Marcuzzi, Sherwin Bahmani, Jiahui Huang, Laura Leal-Taixé and others) · Queryable 3D scenes

LucidLyra

LucidLyra is an NVIDIA research framework that turns a single image into a queryable 3D Gaussian scene encoding geometry, appearance, open-vocabulary semantics and class-agnostic object instances. A frozen camera-controlled video model generates multi-view latents, and a Gaussian decoder combines these with camera rays and C-RADIO image features to predict the scene in one feed-forward pass. During training, frozen SAM/SAM2 and projected SigLIP2 teachers extract object tracks and language-aligned features from the generated videos, so no manual 3D annotations or captured multi-view data are needed. The scenes support novel-view rendering, open-vocabulary segmentation and instance grouping without per-scene optimisation, and the method extends to monocular video to reconstruct dynamic 3D scenes. The authors report that it outperforms baselines pairing 3DGS segmentation methods with generative 3D reconstruction on synthetic and real-image-conditioned benchmarks, without compromising reconstruction quality; no figures are given in the text. Demonstrated uses include robot simulation in Isaac Sim, 3D scene editing and 3D asset mining. The paper is arXiv:2610.40495; the text does not state a licence or code release.