LQGDNet: A Local Quaternion and Global Deep Network for Facial Depression Recognition

Recent visual-based depression recognition methods mostly use hand-crafted features with information lost in color channels, or deep network features with a limited performance from the finite data. In this paper, we propose a method called Local Quaternion and Global Deep Network (LQGDNet) which can combine advantages from hand-crafted and deep features. Specifically, the Quaternion XOR Asymmetrical Regional Local Gradient Coding (XOR-AR-LGC) is first designed, which encodes the facial images with local textures in the quaternion domain to keep the dependence of color channels, and integrated into the Quaternion Feature Extractor (QFE). To the best of our knowledge, it is the first attempt to use a quaternion-based method for facial depression recognition. Second, we design the Local Quaternion Representation Module (LQRM) composed of Local Deep Feature Extractor (LDFE) and QFE to output local quaternion facial features. Third, global deep facial features are encoded from the Global Deep Representation Module (GDRM) with the deep convolutional neural network. Finally, the LQGDNet integrates LQRM and GDRM with the local quaternion and global deep features and predicts the depression score. The experimental results on AVEC 2013 and AVEC 2014 show the superiority of our method compared to the state-of-the-art approaches.

Automated summary

In this paper, a method called LQGDNet is proposed to combine the advantages of hand-crafted and deep features for depression recognition. This method is the first attempt to use a quaternion-based method for facial depression recognition and shows superior performance compared to existing methods in experiments.