期刊
2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021
卷 -, 期 -, 页码 8645-8654出版社
IEEE COMPUTER SOC
DOI: 10.1109/CVPR46437.2021.00854
关键词
-
资金
- TUM-IAS Rudolf Mossbauer Fellowship
- ERC [804724]
- German Research Foundation (DFG)
- European Research Council (ERC) [804724] Funding Source: European Research Council (ERC)
Dynamic neural radiance fields are introduced for modeling appearance and dynamics of a human face, utilizing an implicit representation based on scene representation networks and combining with a low-dimensional morphable model to handle facial dynamics. The volumetric representation allows for photorealistic image generation surpassing current video-based reenactment methods.
We present dynamic neural radiance fields for modeling the appearance and dynamics of a human face(1). Digitally modeling and reconstructing a talking human is a key building-block for a variety of applications. Especially, for telepresence applications in AR or VR, a faithful reproduction of the appearance including novel viewpoint or headposes is required. In contrast to state-of-the-art approaches that model the geometry and material properties explicitly, or are purely image-based, we introduce an implicit representation of the head based on scene representation networks. To handle the dynamics of the face, we combine our scene representation network with a low-dimensional morphable model which provides explicit control over pose and expressions. We use volumetric rendering to generate images from this hybrid representation and demonstrate that such a dynamic neural scene representation can be learned from monocular input data only, without the need of a specialized capture setup. In our experiments, we show that this learned volumetric representation allows for photorealistic image generation that surpasses the quality of state-of-the-art video-based reenactment methods.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据