Coupled adversarial learning for semi-supervised heterogeneous face recognition

Visible-near infrared (VIS-NIR) face matching is a challenging issue in heterogeneous face recognition due to the large spectrum domain discrepancy as well as the over-fitting on insufficient pairwise VIS and NIR images during training. This paper proposes a coupled adversarial learning (CAL) approach for the VISNIR face matching by performing adversarial learning on both image and feature levels. On the image level, we learn a transformation network from unpaired NIR-VIS images to transform a NIR image to VIS domain. Cycle loss, global intensity loss and local texture loss are employed to better capture the discrepancy between NIR and VIS domains. The synthesized NIR or VIS images can be further used to alleviate the over-fitting problem in a semi-supervised way. On the feature level, we seek a shared feature space in which the heterogeneous face matching problem can be approximately treated as a homogeneous face matching problem. An adversarial loss and an orthogonal constraint are employed to reduce the spectrum domain discrepancy and the over-fitting problem, respectively. Experimental results show that CAL not only synthesizes high-quality VIS or NIR images, but also obtains state-of-the-art recognition results. (c) 2020 Elsevier Ltd. All rights reserved.

Automated summary

This study introduces a coupled adversarial learning (CAL) approach to address the challenging issue of VIS-NIR face matching, by conducting adversarial learning on both image and feature levels. Experimental results demonstrate that CAL not only synthesizes high-quality VIS or NIR images, but also achieves state-of-the-art recognition results.