GraphPhys: Facial video-based physiological measurement with graph neural network

Non-contact facial video-based physiological signal measurement using remote photoplethysmography (rPPG) has shown great potential in many applications (e.g. telemedicine, driver fatigue detection, sleep detection) and has achieved reliable results in the controlled environment. With the development of deep learning, more and more methods have been proposed. However, these methods were usually implemented based on model variations that have achieved better results in computer vision tasks, and finally chose average pooling layer or one-dimensional convolution layer directly reduce dimension. Such these operations ignored the fact that the physiological signals of one individual are consistent in different facial regions. In this paper, we propose a novel remote photoplethysmography framework that utilizes graph neural network to extract physiological signal, named GraphPhys. And we design the Average Relative GraphConv based on the fact that the physiological signals of one individual are always consistent in different facial regions for the remote physiological signal measurement task. We apply GraphPhys to two different architectures for rPPG methods, and the methods based on GraphPhys significantly outperform the original methods on the VIPL-HR (Niu et al., 2019) dataset. To the best of our knowledge, this is the first work to utilize graph neural network to aggregate physiological signals. The code is available at https://github.com/XiongJiaHao/GraphPhys.

Automated summary

This paper proposes a novel remote photoplethysmography framework, named GraphPhys, which utilizes graph neural network to extract physiological signals and introduces Average Relative GraphConv for the task of remote physiological signal measurement. Experimental results show that the methods based on GraphPhys significantly outperform the original methods.