Improving Tail Performance Of A Deliberation E2E ASR Model Using A Large Text Corpus

Abstract

End-to-end (E2E) automatic speech recognition (ASR) systems lack the distinct language model (LM) component that characterizes traditional speech systems. While this simplifies the model architecture, it complicates the task of incorporating text-only data into training, which is important to the recognition of tail words that do not occur often in audio-text pairs. While shallow fusion has been proposed as a method for incorporating a pre-trained LM into an E2E model at inference time, it has not yet been explored for very large text corpora, and it has been shown to be very sensitive to hyperparameter settings in the beam search. In this work, we apply shallow fusion to incorporate a very large text corpus into a state-of-the-art E2EASR model. We explore the impact of model size and show that intelligent pruning of the training set can be more effective than increasing the parameter count. Additionally, we show that incorporating the LM in minimum word error rate (MWER) fine tuning m

Stats

• citations22
• S2 citations—
• github stars0
• HF likes0
• heat score10.21
• arxiv keypeyser2020improving

Related papers

• Multilingual And Fully Non-autoregressive ASR With Large Language Model Fusion: A Comprehensive Study (2024)0.00
• Adapting Large Language Model With Speech For Fully Formatted End-to-end Speech Recognition (2023)0.00
• Integrating Pre-trained Speech And Language Models For End-to-end Speech Recognition (2023)0.00
• Delayed Fusion: Integrating Large Language Models Into First-pass Decoding In End-to-end Speech Recognition (2025)5.84
• Effective Text Adaptation For Llm-based ASR Through Soft Prompt Fine-tuning (2024)5.84
• A Comprehensive Solution To Connect Speech Encoder And Large Language Model For ASR (2024)0.00
• Memory Augmented Lookup Dictionary Based Language Modeling For Automatic Speech Recognition (2022)0.00
• Integrating Text Inputs For Training And Adapting RNN Transducer ASR Models (2022)9.59