Multilingual Convolutional, Long Short-Term Memory, Deep Neural Networks for Low Resource Speech Recognition

Stand-alone and the combined model of Convolutional Neural networks (CNNs) and Long Short-Term Memory (LSTM) and Deep neural Networks (DNNs) have shown great improvements in a variety of Speech Recognition tasks. In this paper we also combined these networks but in this paper we used them for multilingual speech recognition, for the prediction and correction (PAC) architecture, in order to calculate the state probability. Our proposed model is known as PAC-MCLDNN. In this paper, we present experiment results for multilingual training on AP16-OLR task. Furthermore, cross-lingual model transfer and multitask learning for under resourced languages such as Uyghur and Vietnam are also performed which further improved the recognition results.