High-resolution facial expression image restoration via adaptive total variation regularization for classroom learning environment

Infrared videos play an important role in recording the learning process. Facial expression images are an important part of infrared videos. High-resolution facial images can reflect the emotion of students or teachers in the classroom. However, recorded infrared videos inevitably have random noise and image blur, which influence facial expression recognition and head pose estimation. In this study, we introduce a blind image restoration method with wavelet transform and total variation regularization. The difference between the low-resolution facial expression image and high-resolution one is revealed by the wavelet transform and total variation regularization. The distribution of the wavelet transform coefficient of high-resolution images is sparser than the coefficient distribution of original low-resolution images. The major novelty of this work is that the sparsity of coefficient distribution is described by the wavelet transform and total variation regularization. Furthermore, the proposed method is conducted on real facial images to verify the effectiveness of priori knowledge. Numerical experiments demonstrate that the proposed method can recover high-resolution facial image and facilitate the application on facial expression recognition.

Automated summary

Infrared videos are important for recording the learning process, and facial expression images are a crucial part of these videos. However, recorded infrared videos often suffer from noise and blur, which negatively impact facial expression recognition and head pose estimation. In this study, we propose a blind image restoration method using wavelet transform and total variation regularization. Our method utilizes the sparsity of high-resolution facial images revealed by wavelet transform and total variation regularization. Experimental results on real facial images show the effectiveness of our proposed method in recovering high-resolution images and improving facial expression recognition.