Facial expression recognition based on a multi-task global-local network

Facial expression recognition plays an important role in intelligent human-computer interaction. The clues for understanding facial expressions lie not in global facial appearance, but also in local informative dynamics among different but confusing expressions. In this paper, we design a multi-task learning framework for global-local representation of facial expressions. First, a shared shallow module is designed to learn information from local regions and the global image. Then we construct a part-based module, which processes critical local regions including the eyes, the nose, and the mouth to extract local informative dynamics related to facial expressions. A global face module is proposed to extract global appearance features related to expressions. The proposed network extracts both local-global and spatio-temporal information for a discriminative and robust representation of facial expressions. Through properly fusing these modules into a system, we have achieved competitive results on the CK+ and Oulu-CASIA databases. (c) 2020 Elsevier B.V. All rights reserved.