Micro-expression action unit detection with spatial and channel attention

Action Unit (AU) detection plays an important role in facial behaviour analysis. In the literature, AU detection has extensive researches in macro-expressions. However, to the best of our knowledge, there is limited research about AU analysis for micro-expressions. In this paper, we focus on AU detection in micro-expressions. Due to the small quantity and low intensity of micro-expression databases, micro expression AU detection becomes challenging. To alleviate these problems, in this work, we propose a novel micro-expression AU detection method by utilizing self high-order statistics of spatio-wise and channel-wise features which can be considered as spatial and channel attentions, respectively. Through such spatial attention module, we expect to utilize rich relationship information of facial regions to increase the AU detection robustness on limited micro-expression samples. In addition, considering the low intensity of micro-expression AUs, we further propose to explore high-order statistics for better capturing subtle regional changes on face to obtain more discriminative AU features. Intensive experiments show that our proposed approach outperforms the basic framework by 0.0859 on CASME II, 0.0485 on CASME, and 0.0644 on SAMM in terms of the average F1-score. (c) 2021 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

Automated summary

This paper focuses on AU detection in micro-expressions, proposing a novel method that utilizes self high-order statistics of spatio-wise and channel-wise features as spatial and channel attentions. By leveraging rich relationship information of facial regions through a spatial attention module, the method aims to increase AU detection robustness on limited micro-expression samples. Additionally, exploring high-order statistics for capturing subtle regional changes on face to obtain more discriminative AU features contributes to the improved performance of the proposed approach on CASME II, CASME, and SAMM datasets.