Internal Language Model Estimation For Domain-adaptive End-to-end Speech Recognition

Abstract

The external language models (LM) integration remains a challenging task for end-to-end (E2E) automatic speech recognition (ASR) which has no clear division between acoustic and language models. In this work, we propose an internal LM estimation (ILME) method to facilitate a more effective integration of the external LM with all pre-existing E2E models with no additional model training, including the most popular recurrent neural network transducer (RNN-T) and attention-based encoder-decoder (AED) models. Trained with audio-transcript pairs, an E2E model implicitly learns an internal LM that characterizes the training data in the source domain. With ILME, the internal LM scores of an E2E model are estimated and subtracted from the log-linear interpolation between the scores of the E2E model and the external LM. The internal LM scores are approximated as the output of an E2E model when eliminating its acoustic components. ILME can alleviate the domain mismatch between training and testi

Stats

• citations61
• S2 citations—
• github stars0
• HF likes0
• heat score13.44
• arxiv keymeng2020internal

Related papers

• Internal Language Model Training For Domain-adaptive End-to-end Speech Recognition (2021)11.39
• Internal Language Model Estimation Based Adaptive Language Model Fusion For Domain Adaptation (2022)0.00
• Mask The Bias: Improving Domain-adaptive Generalization Of Ctc-based ASR With Internal Language Model Estimation (2023)3.58
• Investigating Methods To Improve Language Model Integration For Attention-based Encoder-decoder ASR Models (2021)0.00
• Internal Language Model Estimation Based Language Model Fusion For Cross-domain Code-switching Speech Recognition (2022)0.00
• Internal Language Model Estimation Through Explicit Context Vector Learning For Attention-based Encoder-decoder ASR (2022)7.50
• An Empirical Study Of Language Model Integration For Transducer Based Speech Recognition (2022)3.58
• Adaptable End-to-end ASR Models Using Replaceable Internal Lms And Residual Softmax (2023)0.00