Journal
NEURAL COMPUTING & APPLICATIONS
Volume 34, Issue 18, Pages 15603-15615Publisher
SPRINGER LONDON LTD
DOI: 10.1007/s00521-022-07249-7
Keywords
Audiovisual speech recognition; Dlib; Feed forward neural network; Kannada Language; LSTM; MFCC
Categories
Ask authors/readers for more resources
This study develops an audiovisual speech recognition system for the Kannada language, utilizing the mechanisms of audio and visual speech as well as their integration. Through training and testing, the integrated system achieves a high level of accuracy.
Audiovisual speech recognition is one of the promising technologies in a noisy environment. In this work, we develop the database for Kannada Language and develop an AVSR system for the same. The proposed work is categorized into three main components: a. Audio mechanism. b. Visual speech mechanism. c. Integration of audio and visual mechanisms. In the audio model, MFCC is used to extract the features and a one-dimensional convolutional neural network is used for classification. In the visual module, Dlib is used to extract the features and long short-term memory recurrent neural network is used for classification. Finally, integration of audio and visual module is done using feed forward neural network. Audio speech recognition of Kannada dataset training accuracy achieved is 93.86 and 91.07% for testing data using seventy epochs. Visual speech recognition for Kannada dataset training accuracy is 77.57%, and testing accuracy is 75%. After integration, audiovisual speech recognition for Kannada dataset train accuracy is 93.33% and for testing is 92.26%.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available