Learning a bi-level adversarial network with global and local perception for makeup-invariant face verification

Makeup is widely used to improve facial attractiveness and is well accepted by the public. However, different makeup styles will result in significant facial appearance changes. It remains a challenging problem to match makeup and non-makeup face images. This paper proposes a learning from generation approach for makeup-invariant face verification by introducing a bi-level adversarial network (BLAN). To alleviate the negative effects from makeup, we first generate non-makeup images from makeup ones, and then use the synthesized non-makeup images for further verification. Specifically, there are two adversarial sub-networks on different levels in BLAN, with the one on pixel level for reconstructing appealing facial images and the other on feature level for preserving identity information. For the non-makeup image generation module, a two-path network that involves both global and local structures is applied to improve the synthesis quality. Moreover, we make the generator well constrained by incorporating multiple perceptual losses. All the modules are embedded in an end-to-end network and jointly reduce the sensing gap between makeup and non-makeup images. Experimental results on three benchmark makeup face datasets demonstrate that our method achieves state-of-the-art verification accuracy across makeup status and can produce photo-realistic non-makeup face images. (C) 2019 Elsevier Ltd. All rights reserved.