Minimum Bayes Risk Training Of Rnn-transducer For End-to-end Speech Recognition

Abstract

In this work, we propose minimum Bayes risk (MBR) training of RNN-Transducer (RNN-T) for end-to-end speech recognition. Specifically, initialized with a RNN-T trained model, MBR training is conducted via minimizing the expected edit distance between the reference label sequence and on-the-fly generated N-best hypothesis. We also introduce a heuristic to incorporate an external neural network language model (NNLM) in RNN-T beam search decoding and explore MBR training with the external NNLM. Experimental results demonstrate an MBR trained model outperforms a RNN-T trained model substantially and further improvements can be achieved if trained with an external NNLM. Our best MBR trained system achieves absolute character error rate (CER) reductions of 1.2% and 0.5% on read and spontaneous Mandarin speech respectively over a strong convolution and transformer based RNN-T baseline trained on ~21,000 hours of speech.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyweng2019minimum

Related papers

• Efficient Minimum Word Error Rate Training Of Rnn-transducer For End-to-end Speech Recognition (2020)11.19
• Improving RNN Transducer Modeling For End-to-end Speech Recognition (2019)0.00
• Minimum Bayes Risk Training For End-to-end Speaker-attributed ASR (2020)0.00
• Exploring Rnn-transducer For Chinese Speech Recognition (2018)9.23
• Multiple-hypothesis RNN-T Loss For Unsupervised Fine-tuning And Self-training Of Neural Transducer (2022)0.00
• Multitask Learning And Joint Optimization For Transformer-rnn-transducer Speech Recognition (2020)8.09
• Research On Modeling Units Of Transformer Transducer For Mandarin Speech Recognition (2020)0.00
• Exploring Pre-training With Alignments For RNN Transducer Based End-to-end Speech Recognition (2020)9.41