Abstract
We propose Ref-GeNVS, a training-free, reflection-aware method for generative novel view synthesis (NVS) in mirror scenes. Existing multi-view diffusion models often fail to recognize the mirror in the scene and cannot exploit reflected content for scene generation. To fix this issue without additional training, our key idea is to treat a mirror image as two complementary views. From input images, we estimate the mirror plane and reflect camera poses to form virtual views. Based on this virtual view setup, we propose a two-stage generation method consisting of Mirror-gated attention and Reflection injection, which enables reflection-consistent NVS by explicitly leveraging reflection relationships in a multi-view diffusion model.
Ref-GeNVS inherits the strong generalizability of the multi-view diffusion backbone, while it does not require finetuning. On synthetic and real scenes including mirrors, Ref-GeNVS outperforms recent generative NVS methods by generating reflection-consistent and contextually coherent novel views, revealing scene structure visible only through mirrors.
Key Idea: Reflected View Acquisition & Conditioning
We treat a mirror image as two complementary views to generate the scene reflected on the mirror. To use reflections as auxiliary cues, we first estimate the plane equation of the mirror in the scene, and reflect camera poses of input views to the mirror to construct virtual reflected views.
Training-free Two-stage Pipeline
Stage 1 generates target views with mirror-masked input images and reflected virtual views while restricting conditioning to the mirror regions of the reflected views using Mirror-gated attention. Stage 2 completes the mirror surface via Reflection injection by injecting features from the reflected target poses into the mirror region at each denoising step.
