Supervised Pixel-Wise GAN for Face Super-Resolution

For many face-related multimedia applications, low-resolution face images may greatly degrade the face recognition performance and necessitate face super-resolution (SR). Among the current SR methods, MSE-oriented SR methods often produce over-smoothed outputs and could miss some texture details while GAN-oriented SR methods may generate artifacts which are harmful to face recognition. To resolve the above issues, this paper presents a supervised pixel-wise Generative Adversarial Network (SPGAN) that can resolve a very low-resolution face image of 16 x 16 or smaller pixel-size to its larger version of multiple scaling factors (2x, 4x, 8x and even 16x) in a unified framework. Being different from traditional unsupervised discriminators which generate a single number to represent the likelihood whether the input image is real or fake, the proposed supervised pixel-wise discriminator mainly focus on whether each pixel of the generated SR face image is as photo-realistic as its corresponding pixel in the ground-truth HR (high-resolution) face image. To further improve the face recognition performance of SPGAN, we take advantage of the face identity prior by sending two inputs to the discriminator, including an input face image (either a real HR face image or its corresponding SR face image) and its face features which are extracted from a pre-trained face recognition model. Due to the introduced face identity prior, the identity-based discriminator can pay more attention to texture details which are closely related to face recognition. Extensive experiments demonstrate that the proposed SPGAN can achieve more photo-realistic SRimages and higher face recognition accuracy than some state-of-the-art methods.

Automated summary

The paper introduces a supervised pixel-wise Generative Adversarial Network (SPGAN) that can upscale low-resolution face images to larger versions with multiple scaling factors. By utilizing face features and identity prior, SPGAN enhances face recognition performance by focusing on texture details. Extensive experiments show that SPGAN produces more photo-realistic super-resolution images and better face recognition accuracy compared to state-of-the-art methods.