A Spatio-Temporal Spotting Network with Sliding Windows for Micro-Expression Detection

Micro-expressions reveal underlying emotions and are widely applied in political psychology, lie detection, law enforcement and medical care. Micro-expression spotting aims to detect the temporal locations of facial expressions from video sequences and is a crucial task in micro-expression recognition. In this study, the problem of micro-expression spotting is formulated as micro-expression classification per frame. We propose an effective spotting model with sliding windows called the spatio-temporal spotting network. The method involves a sliding window detection mechanism, combines the spatial features from the local key frames and the global temporal features and performs micro-expression spotting. The experiments are conducted on the CAS(ME)2 database and the SAMM Long Videos database, and the results demonstrate that the proposed method outperforms the state-of-the-art method by 30.58% for the CAS(ME)2 and 23.98% for the SAMM Long Videos according to overall F-scores.

Automated summary

This study investigates the problem of micro-expression spotting as a frame-by-frame micro-expression classification problem and proposes an effective spotting model. The experimental results demonstrate that the proposed method outperforms the state-of-the-art method in terms of overall F-scores on the CAS(ME)2 and SAMM Long Videos databases.