Exploring Architectures, Data And Units For Streaming End-to-end Speech Recognition With Rnn-transducer

Abstract

We investigate training end-to-end speech recognition models with the recurrent neural network transducer (RNN-T): a streaming, all-neural, sequence-to-sequence architecture which jointly learns acoustic and language model components from transcribed acoustic data. We explore various model architectures and demonstrate how the model can be improved further if additional text or pronunciation data are available. The model consists of an `encoder', which is initialized from a connectionist temporal classification-based (CTC) acoustic model, and a `decoder' which is partially initialized from a recurrent neural network language model trained on text data alone. The entire neural network is trained with the RNN-T loss and directly outputs the recognized transcript as a sequence of graphemes, thus performing end-to-end speech recognition. We find that performance can be improved further through the use of sub-word units (`wordpieces') which capture longer context and significantly reduce su

Stats

• citations144
• S2 citations—
• github stars0
• HF likes0
• heat score16.21
• arxiv keyrao2018exploring

Related papers

• Improving RNN Transducer Modeling For End-to-end Speech Recognition (2019)0.00
• Transformer Transducer: A Streamable Speech Recognition Model With Transformer Encoders And RNN-T Loss (2020)18.58
• Exploring Neural Transducers For End-to-end Speech Recognition (2017)14.90
• Exploring Pre-training With Alignments For RNN Transducer Based End-to-end Speech Recognition (2020)9.41
• Cascade Rnn-transducer: Syllable Based Streaming On-device Mandarin Speech Recognition With A Syllable-to-character Converter (2020)9.92
• Improved Neural Language Model Fusion For Streaming Recurrent Neural Network Transducer (2020)8.82
• Exploring Rnn-transducer For Chinese Speech Recognition (2018)9.23
• Minimum Latency Training Of Sequence Transducers For Streaming End-to-end Speech Recognition (2022)0.00