A Comparison Of Techniques For Language Model Integration In Encoder-decoder Speech Recognition
2018 · Shubham Toshniwal, Anjuli Kannan, Chung-Cheng Chiu, et al.
Abstract
Attention-based recurrent neural encoder-decoder models present an elegant solution to the automatic speech recognition problem. This approach folds the acoustic model, pronunciation model, and language model into a single network and requires only a parallel corpus of speech and text for training. However, unlike in conventional approaches that combine separate acoustic and language models, it is not clear how to use additional (unpaired) text. While there has been previous work on methods addressing this problem, a thorough comparison among methods is still lacking. In this paper, we compare a suite of past methods and some of our own proposed methods for using unpaired text data to improve encoder-decoder models. For evaluation, we use the medium-sized Switchboard data set and the large-scale Google voice search and dictation data sets. Our results confirm the benefits of using unpaired text across a range of methods and data sets. Surprisingly, for first-pass decoding, the rather s
Authors
(none)
Tags
Stats
Related papers
- Investigating Methods To Improve Language Model Integration For Attention-based Encoder-decoder ASR Models (2021)0.00
- On Decoder-only Architecture For Speech-to-text And Large Language Model Integration (2023)0.00
- Optimizing Alignment Of Speech And Language Latent Spaces For End-to-end Speech Recognition And Understanding (2021)9.03
- Prepending Or Cross-attention For Speech-to-text? An Empirical Comparison (2025)2.26
- On The Comparison Of Popular End-to-end Models For Large Scale Speech Recognition (2020)0.00
- Towards Better Decoding And Language Model Integration In Sequence To Sequence Models (2016)15.67
- State-of-the-art Speech Recognition With Sequence-to-sequence Models (2017)21.01
- Decoupling Pronunciation And Language For End-to-end Code-switching Automatic Speech Recognition (2020)0.00