Journal
IEEE TRANSACTIONS ON MULTIMEDIA
Volume 24, Issue -, Pages 4028-4040Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TMM.2021.3111485
Keywords
Image reconstruction; Three-dimensional displays; Hair; Solid modeling; Semantics; Faces; Rendering (computer graphics); 3D head reconstruction; multi-view stereo; prior guidance; neural rendering
Categories
Funding
- National Natural Science Foundation of China [62122071]
- Youth Innovation Promotion Association CAS [2018495]
- Fundamental Research Funds for the Central Universities [WK3470000021]
Ask authors/readers for more resources
In this paper, a prior-guided implicit neural rendering network is proposed to recover a high-fidelity 3D head model using a few multi-view portrait images as input. By utilizing human head priors including facial prior knowledge, head semantic segmentation information, and 2D hair orientation maps, the proposed method achieves improved reconstruction accuracy and robustness.
Recovery of a 3D head model including the complete face and hair regions is still a challenging problem in computer vision and graphics. In this paper, we consider this problem using only a few multi-view portrait images as input. Previous multi-view stereo methods that have been based, either on optimization strategies or deep learning techniques, suffer from low-frequency geometric structures such as unclear head structures and inaccurate reconstruction in hair regions. To tackle this problem, we propose a prior-guided implicit neural rendering network. Specifically, we model the head geometry with a learnable signed distance field (SDF) and optimize it via an implicit differentiable renderer with the guidance of some human head priors, including the facial prior knowledge, head semantic segmentation information and 2D hair orientation maps. The utilization of these priors can improve the reconstruction accuracy and robustness, leading to a high-quality integrated 3D head model. Extensive ablation studies and comparisons with state-of-the-art methods demonstrate that our method can generate high-fidelity 3D head geometries with the guidance of these priors.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available