4-bit Quantization of LSTM-based Speech Recognition Models

We investigate the impact of aggressive low-precision representations of weights and activations in two families of large LSTM-based architectures for Automatic Speech Recognition (ASR): hybrid Deep Bidirectional LSTM - Hidden Markov Models (DBLSTM-HMMs) and Recurrent Neural Network - Transducers (RNN-Ts). Using a 4-bit integer representation, a naive quantization approach applied to the LSTM portion of these models results in significant Word Error Rate (WER) degradation. On the other hand, we show that minimal accuracy loss is achievable with an appropriate choice of quantizers and initializations. In particular, we customize quantization schemes depending on the local properties of the network, improving recognition performance while limiting computational time. We demonstrate our solution on the Switchboard (SWB) and CallHome (CH) test sets of the NIST Hub5-2000 evaluation. DBLSTM-HMMs trained with 300 or 2000 hours of SWB data achieves < 0.5% and < 1% average WER degradation, respectively. On the more challenging RNN-T models, our quantization strategy limits degradation in 4-bit inference to 1.3%.

Automated summary

The study investigates the impact of aggressive low-precision representations on two families of large LSTM-based architectures for Automatic Speech Recognition. It shows that with appropriate choice of quantizers and initializations, minimal accuracy loss can be achieved while improving recognition performance and limiting computational time. The quantization strategy demonstrated in the study limits degradation in 4-bit inference to 1.3% on challenging RNN-T models.