Re-Ranking High-Dimensional Deep Local Representation for NIR-VIS Face Recognition

Heterogeneous face recognition refers to matching facial images captured from different sensors or sources, which has wide applications in public security and law enforcement. Because of the great differences in sensing and creating procedure, there is a huge feature gap between heterogeneous facial images. The existing methods merely focus on comparing the probe image with the gallery in feature space, while the true target may not appear at the first rank due to the appearance variations caused by different sensing patterns. In order to exploit valuable information from the initial ranking result, this paper proposes to re-rank high-dimensional deep local representation for matching near-infrared (NIR) and visual (VIS) facial images, i.e., NIR-VIS face recognition. A high-dimensional deep local representation is first constructed by extracting and concatenating deep features on local facial patches via a convolutional neural network (CNN). The initial NIR-VIS recognition ranking results can be obtained by comparing the compressed deep features. We then propose a novel and efficient locally linear re-ranking (LLRe-Rank) technique to refine the initial ranking results, which can explore valuable information from the initial ranking result. The proposed re-ranking method does not require any human interaction or data annotation and can be served as an unsupervised postprocessing technique. The experimental results on the most challenging Oulu-CASIA NIR-VIS database and CASIA NIR-VIS 2.0 database demonstrate the effectiveness of our method.