Journal
Publisher
ISCA-INT SPEECH COMMUNICATION ASSOC
DOI: 10.21437/Interspeech.2016-595
Keywords
neural networks; sequence discriminative training
Categories
Funding
- NSF CRI Grant [1513128]
- DARPA LORELEI Contract [HR0011-15-2-0024]
- IARPA BABEL Contract [2012-12050800010]
- Direct For Computer & Info Scie & Enginr
- Division Of Computer and Network Systems [1513128] Funding Source: National Science Foundation
Ask authors/readers for more resources
In this paper we describe a method to perform sequence discriminative training of neural network acoustic models without the need for frame-level cross-entropy pre-training. We use the lattice-free version of the maximum mutual information (MMI) criterion: LF-MMI. To make its computation feasible we use a phone n-gram language model, in place of the word language model. To further reduce its space and time complexity we compute the objective function using neural network outputs at one third the standard frame rate. These changes enable us to perform the computation for the forward-backward algorithm on GPUs. Further the reduced output frame-rate also provides a significant speed-up during decoding. We present results on 5 different LVCSR tasks with training data ranging from 100 to 2100 hours. Models trained with LF-MMI provide a relative word error rate reduction of similar to 11.5%, over those trained with cross-entropy objective function, and similar to 8%, over those trained with cross-entropy and sMBR objective functions. A further reduction similar to 2.5%of relative, can be obtained by fine tuning these models with the word-lattice based sMBR objective function.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available