A Comparative Study On Transformer Vs RNN In Speech Applications

Abstract

Sequence-to-sequence models have been widely used in end-to-end speech processing, for example, automatic speech recognition (ASR), speech translation (ST), and text-to-speech (TTS). This paper focuses on an emergent sequence-to-sequence model called Transformer, which achieves state-of-the-art performance in neural machine translation and other natural language processing applications. We undertook intensive studies in which we experimentally compared and analyzed Transformer and conventional recurrent neural networks (RNN) in a total of 15 ASR, one multilingual ASR, one ST, and two TTS benchmarks. Our experiments revealed various training tips and significant performance benefits obtained with Transformer for each task including the surprising superiority of Transformer in 13/15 ASR benchmarks in comparison with RNN. We are preparing to release Kaldi-style reproducible recipes using open source and publicly available datasets for all the ASR, ST, and TTS tasks for the community to su

Stats

• citations473
• S2 citations—
• github stars0
• HF likes0
• heat score20.07
• arxiv keykarita2019a

Related papers

• Exploring Neural Transducers For End-to-end Speech Recognition (2017)14.90
• On The Comparison Of Popular End-to-end Models For Large Scale Speech Recognition (2020)0.00
• Multitask Learning And Joint Optimization For Transformer-rnn-transducer Speech Recognition (2020)8.09
• Attention Is All You Need In Speech Separation (2020)20.59
• A Comparison Of End-to-end Models For Long-form Speech Recognition (2019)12.93
• Fast Offline Transformer-based End-to-end Automatic Speech Recognition For Real-world Applications (2021)7.16
• Improving RNN Transducer Based ASR With Auxiliary Tasks (2020)9.59
• Transformer-transducer: End-to-end Speech Recognition With Self-attention (2019)0.00