Attention-based ASR With Lightweight And Dynamic Convolutions

Abstract

End-to-end (E2E) automatic speech recognition (ASR) with sequence-to-sequence models has gained attention because of its simple model training compared with conventional hidden Markov model based ASR. Recently, several studies report the state-of-the-art E2E ASR results obtained by Transformer. Compared to recurrent neural network (RNN) based E2E models, training of Transformer is more efficient and also achieves better performance on various tasks. However, self-attention used in Transformer requires computation quadratic in its input length. In this paper, we propose to apply lightweight and dynamic convolution to E2E ASR as an alternative architecture to the self-attention to make the computational order linear. We also propose joint training with connectionist temporal classification, convolution on the frequency axis, and combination with self-attention. With these techniques, the proposed architectures achieve better performance than RNN-based E2E model and performance competitiv

Stats

• citations15
• S2 citations—
• github stars0
• HF likes0
• heat score9.03
• arxiv keyfujita2019attention

Related papers

• Transformer-based Online Ctc/attention End-to-end Speech Recognition Architecture (2020)14.06
• Improving Transformer-based Conversational ASR By Inter-sentential Attention Mechanism (2022)7.50
• Efficient Conformer With Prob-sparse Attention Mechanism For End-to-endspeech Recognition (2021)8.09
• Improving Hybrid Ctc/attention End-to-end Speech Recognition With Pretrained Acoustic And Language Model (2021)8.82
• Towards Online End-to-end Transformer Automatic Speech Recognition (2019)0.00
• Towards A Unified Conformer Structure: From ASR To ASV Task (2022)13.11
• Conv-transformer Transducer: Low Latency, Low Frame Rate, Streamable End-to-end Speech Recognition (2020)11.08
• Transformer-transducer: End-to-end Speech Recognition With Self-attention (2019)0.00