Deep Learning-Based Automated Lip-Reading: A Survey

A survey on automated lip-reading approaches is presented in this paper with the main focus being on deep learning related methodologies which have proven to be more fruitful for both feature extraction and classification. This survey also provides comparisons of all the different components that make up automated lip-reading systems including the audio-visual databases, feature extraction, classification networks and classification schemas. The main contributions and unique insights of this survey are: 1) A comparison of Convolutional Neural Networks with other neural network architectures for feature extraction; 2) A critical review on the advantages of Attention-Transformers and Temporal Convolutional Networks to Recurrent Neural Networks for classification; 3) A comparison of different classification schemas used for lip-reading including ASCII characters, phonemes and visemes, and 4) A review of the most up-to-date lip-reading systems up until early 2021.

Automated summary

This paper presents a survey on automated lip-reading approaches, focusing on deep learning methodologies. The survey compares different components of automated lip-reading systems and highlights the advantages of Convolutional Neural Networks, Attention-Transformers, and Temporal Convolutional Networks. Additionally, it compares different classification schemas used for lip-reading and reviews the most up-to-date lip-reading systems.