Transformer-based Acoustic Modeling For Hybrid Speech Recognition

Abstract

We propose and evaluate transformer-based acoustic models (AMs) for hybrid speech recognition. Several modeling choices are discussed in this work, including various positional embedding methods and an iterated loss to enable training deep transformers. We also present a preliminary study of using limited right context in transformer models, which makes it possible for streaming applications. We demonstrate that on the widely used Librispeech benchmark, our transformer-based AM outperforms the best published hybrid result by 19% to 26% relative when the standard n-gram language model (LM) is used. Combined with neural network LM for rescoring, our proposed approach achieves state-of-the-art results on Librispeech. Our findings are also confirmed on a much larger internal dataset.

Stats

• citations148
• S2 citations—
• github stars0
• HF likes0
• heat score16.30
• arxiv keywang2019transformer

Related papers

• Streaming Transformer-based Acoustic Models Using Self-attention With Augmented Memory (2020)0.00
• Transformers With Convolutional Context For ASR (2019)0.00
• Transformer In Action: A Comparative Study Of Transformer-based Acoustic Models For Large Scale Speech Recognition Applications (2020)9.41
• Transformer Transducer: One Model Unifying Streaming And Non-streaming Speech Recognition (2020)0.00
• Transformer-transducer: End-to-end Speech Recognition With Self-attention (2019)0.00
• Improving Hybrid Ctc/attention End-to-end Speech Recognition With Pretrained Acoustic And Language Model (2021)8.82
• Transformer Language Models With Lstm-based Cross-utterance Information Representation (2021)10.48
• Modular Hybrid Autoregressive Transducer (2022)8.35