EfficientPhys: Enabling Simple, Fast and Accurate Camera-Based Cardiac Measurement

Camera-based physiological measurement is a growing field with neural models providing state-of-the-art performance. Prior research has explored various end-to-end architectures; however these methods still require several preprocessing steps and are not able to run directly on mobile and edge devices. The operations are often non-trivial to implement, making replication and deployment difficult and can even have a higher computational budget than the core network itself. In this paper, we propose two novel and efficient neural models for camera-based physiological measurement called EfficientPhys that remove the need for face detection, segmentation, normalization, color space transformation or any other preprocessing steps. Using an input of raw video frames, our models achieve strong accuracy on three public datasets. We show that this is the case whether using a transformer or convolutional backbone. We further evaluate the latency of the proposed networks and show that our most lightweight network also achieves a 33% improvement in efficiency.

Automated summary

This paper proposes two novel and efficient neural models, called EfficientPhys, for camera-based physiological measurement, which eliminate the need for face detection, segmentation, normalization, color space transformation, or any other preprocessing steps. Our models achieve strong accuracy on three public datasets using raw video frames as input, whether using a transformer or convolutional backbone. Furthermore, we evaluate the latency of the proposed networks and demonstrate a 33% improvement in efficiency for our most lightweight network.