3.8 Proceedings Paper

3D-Aware Multi-Class Image-to-Image Translation with NeRFs

出版社

IEEE COMPUTER SOC
DOI: 10.1109/CVPR52729.2023.01217

关键词

-

向作者/读者索取更多资源

Recent advances in 3D-aware generative models combined with Neural Radiance Fields have achieved impressive results in 3D consistent multi-class image-to-image translation. To address the unrealistic shape/identity change in 2D-I2I translation, the learning process is divided into a multi-class 3D-aware GAN step and a 3D-aware I2I translation step, with novel techniques proposed to reduce view-consistency problems.
Recent advances in 3D-aware generative models (3D-aware GANs) combined with Neural Radiance Fields (NeRF) have achieved impressive results. However no prior works investigate 3D-aware GANs for 3D consistent multi-class image-to-image (3D-aware I2I) translation. Naively using 2D-I2I translation methods suffers from unrealistic shape/identity change. To perform 3D-aware multi-class I2I translation, we decouple this learning process into a multi-class 3D-aware GAN step and a 3D-aware I2I translation step. In the first step, we propose two novel techniques: a new conditional architecture and an effective training strategy. In the second step, based on the well-trained multi-class 3D-aware GAN architecture, that preserves view-consistency, we construct a 3D-aware I2I translation system. To further reduce the view-consistency problems, we propose several new techniques, including a U-net-like adaptor network design, a hierarchical representation constrain and a relative regularization loss. In extensive experiments on two datasets, quantitative and qualitative results demonstrate that we successfully perform 3D-aware I2I translation with multi-view consistency. Code is available in 3DI2I.

作者

我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。

评论

主要评分

3.8
评分不足

次要评分

新颖性
-
重要性
-
科学严谨性
-
评价这篇论文

推荐

暂无数据
暂无数据