Sequence-to-sequence Learning Via Attention Transfer For Incremental Speech Recognition

Abstract

Attention-based sequence-to-sequence automatic speech recognition (ASR) requires a significant delay to recognize long utterances because the output is generated after receiving entire input sequences. Although several studies recently proposed sequence mechanisms for incremental speech recognition (ISR), using different frameworks and learning algorithms is more complicated than the standard ASR model. One main reason is because the model needs to decide the incremental steps and learn the transcription that aligns with the current short speech segment. In this work, we investigate whether it is possible to employ the original architecture of attention-based ASR for ISR tasks by treating a full-utterance ASR as the teacher model and the ISR as the student model. We design an alternative student network that, instead of using a thinner or a shallower model, keeps the original architecture of the teacher model but with shorter sequences (few encoder and decoder states). Using attention

Stats

• citations3
• S2 citations—
• github stars0
• HF likes0
• heat score4.52
• arxiv keynovitasari2020sequence

Related papers

• Integrating Source-channel And Attention-based Sequence-to-sequence Models For Speech Recognition (2019)8.09
• Incremental Text To Speech For Neural Sequence-to-sequence Models Using Reinforcement Learning (2020)5.24
• State-of-the-art Speech Recognition With Sequence-to-sequence Models (2017)21.01
• Improving Transformer-based Conversational ASR By Inter-sentential Attention Mechanism (2022)7.50
• Sequence-to-sequence ASR Optimization Via Reinforcement Learning (2017)9.41
• A Machine Of Few Words -- Interactive Speaker Recognition With Reinforcement Learning (2020)5.24
• Supervised Attention In Sequence-to-sequence Models For Speech Recognition (2022)5.84
• Forward Attention In Sequence-to-sequence Acoustic Modelling For Speech Synthesis (2018)12.10