期刊
PATTERN RECOGNITION LETTERS
卷 166, 期 -, 页码 38-45出版社
ELSEVIER
DOI: 10.1016/j.patrec.2022.12.027
关键词
RGB-D face recognition; Multi-modal fusion; Depth enhancement; Multi-head-attention mechanism; Incomplete modal data
This paper proposes a progressive multi-modal fusion framework to improve and enhance RGB-D facial recognition using low-cost RGB-D cameras. The framework includes a depth enhancement module, feature extraction and aggregation steps, as well as masked modeling and inter-modal feature interaction modules. Experimental results demonstrate the superior performance and robustness of the proposed solution compared to other face recognition methods on four challenging benchmark databases.
Existing RGB-based 2D face recognition approaches are sensitive to facial variations, posture, occlusions, and illumination. Current depth-based methods have been proved to alleviate the above sensitivity by introducing geometric information but rely heavily on high-quality depth from high-cost RGB-D cameras. To this end, we propose a Progressive Multi-modal Fusion framework to exploit enhanced and robust face representation for RGB-D facial recognition based on low-cost RGB-D cameras, which also deals with in-complete RGB-D modal data. Due to the defects such as holes caused by low-cost cameras, we first design a depth enhancement module to refine the low-quality depth and correct depth inaccuracies. Then, we extract and aggregate augmented feature maps of RGB and depth modality step-by-step. Subsequently, the masked modeling scheme and iterative inter-modal feature interaction module aim to fully exploit the implicit relations among these two modalities. We perform comprehensive experiments to verify the superior performance and robustness of the proposed solution over other FR approaches on four chal-lenging benchmark databases. (c) 2022 Elsevier B.V. All rights reserved.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据