End-to-end Speech Recognition With Word-based RNN Language Models

Abstract

This paper investigates the impact of word-based RNN language models (RNN-LMs) on the performance of end-to-end automatic speech recognition (ASR). In our prior work, we have proposed a multi-level LM, in which character-based and word-based RNN-LMs are combined in hybrid CTC/attention-based ASR. Although this multi-level approach achieves significant error reduction in the Wall Street Journal (WSJ) task, two different LMs need to be trained and used for decoding, which increase the computational cost and memory usage. In this paper, we further propose a novel word-based RNN-LM, which allows us to decode with only the word-based LM, where it provides look-ahead word probabilities to predict next characters instead of the character-based LM, leading competitive accuracy with less computation compared to the multi-level LM. We demonstrate the efficacy of the word-based RNN-LMs using a larger corpus, LibriSpeech, in addition to WSJ we used in the prior work. Furthermore, we show that the

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyhori2018end

Related papers

• Multi-task Language Modeling For Improving Speech Recognition Of Rare Words (2020)8.35
• Integrating Text Inputs For Training And Adapting RNN Transducer ASR Models (2022)9.59
• Integrating Pre-trained Speech And Language Models For End-to-end Speech Recognition (2023)0.00
• End-to-end Speech Recognition Contextualization With Large Language Models (2023)0.00
• Improved Neural Language Model Fusion For Streaming Recurrent Neural Network Transducer (2020)8.82
• An Improved Hybrid Ctc-attention Model For Speech Recognition (2018)0.00
• Improving RNN Transducer Modeling For End-to-end Speech Recognition (2019)0.00
• A Comparison Of End-to-end Models For Long-form Speech Recognition (2019)12.93