Transformer-transducer: End-to-end Speech Recognition With Self-attention

Abstract

We explore options to use Transformer networks in neural transducer for end-to-end speech recognition. Transformer networks use self-attention for sequence modeling and comes with advantages in parallel computation and capturing contexts. We propose 1) using VGGNet with causal convolution to incorporate positional information and reduce frame rate for efficient inference 2) using truncated self-attention to enable streaming for Transformer and reduce computational complexity. All experiments are conducted on the public LibriSpeech corpus. The proposed Transformer-Transducer outperforms neural transducer with LSTM/BLSTM networks and achieved word error rates of 6.37 % on the test-clean set and 15.30 % on the test-other set, while remaining streamable, compact with 45.7M parameters for the entire system, and computationally efficient with complexity of O(T), where T is input sequence length.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyyeh2019transformer

Related papers

• Transformer Transducer: A Streamable Speech Recognition Model With Transformer Encoders And RNN-T Loss (2020)18.58
• Conv-transformer Transducer: Low Latency, Low Frame Rate, Streamable End-to-end Speech Recognition (2020)11.08
• Multitask Learning And Joint Optimization For Transformer-rnn-transducer Speech Recognition (2020)8.09
• Exploring Neural Transducers For End-to-end Speech Recognition (2017)14.90
• Self-attention Transducers For End-to-end Speech Recognition (2019)11.93
• Developing Real-time Streaming Transformer Transducer For Speech Recognition On Large-scale Dataset (2020)0.00
• Streaming Transformer-based Acoustic Models Using Self-attention With Augmented Memory (2020)0.00
• Transformer Transducer: One Model Unifying Streaming And Non-streaming Speech Recognition (2020)0.00