4.7 Article

Heterogeneous Face Interpretable Disentangled Representation for Joint Face Recognition and Synthesis

Journal

Publisher

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TNNLS.2021.3071119

Keywords

Face recognition; Task analysis; Feature extraction; Image recognition; Sensors; Semantics; Visualization; Disentanglement; heterogeneous face; interpretable representation; joint recognition and synthesis

Funding

  1. National Key Research and Development Program of China [2018AAA0103202]
  2. National Natural Science Foundation of China [62072356, 62050175, 62036007, 61922066, 61876142, 61806152, 61772402]
  3. Xidian University Intellifusion Joint Innovation Laboratory of Artificial Intelligence
  4. Fundamental Research Funds for the Central Universities
  5. China Post-Doctoral Science Foundation [2018M631124, 2019T120880]
  6. Key Research and Development Program of Shaanxi [2020ZDLGY08-08]
  7. Guangxi Natural Science Foundation Program [2021GXNSFDA075011]

Ask authors/readers for more resources

The article explores learning interpretable representations for complex heterogeneous faces and proposes the HFIDR and M-HFIDR methods for cross-modality recognition and synthesis tasks, achieving efficiency in face recognition and synthesis.
Heterogeneous faces are acquired with different sensors, which are closer to real-world scenarios and play an important role in the biometric security field. However, heterogeneous face analysis is still a challenging problem due to the large discrepancy between different modalities. Recent works either focus on designing a novel loss function or network architecture to directly extract modality-invariant features or synthesizing the same modality faces initially to decrease the modality gap. Yet, the former always lacks explicit interpretability, and the latter strategy inherently brings in synthesis bias. In this article, we explore to learn the plain interpretable representation for complex heterogeneous faces and simultaneously perform face recognition and synthesis tasks. We propose the heterogeneous face interpretable disentangled representation (HFIDR) that could explicitly interpret dimensions of face representation rather than simple mapping. Benefited from the interpretable structure, we further could extract latent identity information for cross-modality recognition and convert the modality factor to synthesize cross-modality faces. Moreover, we propose a multimodality heterogeneous face interpretable disentangled representation (M-HFIDR) to extend the basic approach suitable for the multimodality face recognition and synthesis. To evaluate the ability of generalization, we construct a novel large-scale face sketch data set. Experimental results on multiple heterogeneous face databases demonstrate the effectiveness of the proposed method.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.7
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

No Data Available
No Data Available