FER-CHC: Facial expression recognition with cross-hierarchy contrast

Facial expression recognition (FER) tasks with convolutional neural networks (CNNs) have seen remarkable progress. However, these CNN-based approaches do not well capture detailed and crucial features that can distinguish different facial expressions from a global perspective. There is still much room for improvement in the performance of existing CNN-based models for FER. To address this, we propose a novel cross-hierarchy contrast (CHC) framework called FER-CHC for FER tasks. FER-CHC employs a contrastive learning mechanism to utilize these crucial features in improving the performance of CNN-based models for FER. Specifically, FER-CHC utilizes CHC to regularize the feature learning of the backbone network and enhance global representations of facial expressions. The CHC captures common and differential features from different facial expressions with a cross-hierarchy contrast mechanism. Furthermore, a fusion network globally integrates the features learned from both the backbone network and CHC to learn a more robust feature representation. We conducted comprehensive experiments on six popular datasets: CK+, FER2013, FER+, RAF-DB, AffectNet, and JAFFE. The results show that our proposed FER-CHC achieves state-of-the-art performances on these datasets. Additionally, an ablation study was conducted to demonstrate the effectiveness of the proposed components in FER-CHC.& COPY; 2023 Elsevier B.V. All rights reserved.

Automated summary

This study proposes a novel cross-hierarchy contrast (CHC) framework called FER-CHC for facial expression recognition tasks. FER-CHC utilizes a contrastive learning mechanism to improve the performance of CNN-based models by utilizing crucial features and globally integrating them through a fusion network. Experimental results demonstrate that FER-CHC achieves state-of-the-art performances on multiple datasets.