Learning High-Fidelity Face Texture Completion without Complete Face Texture

For face texture completion, previous methods typically use some complete textures captured by multiview imaging systems or 3D scanners for supervised learning. This paper deals with a new challenging problem - learning to complete invisible texture in a single face image without using any complete texture. We simply leverage a large corpus of face images of different subjects (e. g., FFHQ) to train a texture completion model in an unsupervised manner. To achieve this, we propose DSD-GAN, a novel deep neural network based method that applies two discriminators in UV map space and image space. These two discriminators work in a complementary manner to learn both facial structures and texture details. We show that their combination is essential to obtain high-fidelity results. Despite the network never sees any complete facial appearance, it is able to generate compelling full textures from single images.

Automated summary

This study introduces a new method for completing invisible textures in single face images without using any complete textures, achieved through unsupervised learning using a large corpus of face images. The proposed DSD-GAN method utilizes two discriminators in UV map space and image space to learn both facial structures and texture details in a complementary manner, demonstrating the importance of their combination for high-fidelity results. Despite never seeing complete facial appearances, the network is able to generate compelling full textures from single images.