Supervised Attention In Sequence-to-sequence Models For Speech Recognition

Abstract

Attention mechanism in sequence-to-sequence models is designed to model the alignments between acoustic features and output tokens in speech recognition. However, attention weights produced by models trained end to end do not always correspond well with actual alignments, and several studies have further argued that attention weights might not even correspond well with the relevance attribution of frames. Regardless, visual similarity between attention weights and alignments is widely used during training as an indicator of the models quality. In this paper, we treat the correspondence between attention weights and alignments as a learning problem by imposing a supervised attention loss. Experiments have shown significant improved performance, suggesting that learning the alignments well during training critically determines the performance of sequence-to-sequence models.

Stats

• citations5
• S2 citations—
• github stars0
• HF likes0
• heat score5.84
• arxiv keyyang2022supervised

Related papers

• Multi-scale Alignment And Contextual History For Attention Mechanism In Sequence-to-sequence Model (2018)7.16
• Forward Attention In Sequence-to-sequence Acoustic Modelling For Speech Synthesis (2018)12.10
• On Using 2D Sequence-to-sequence Models For Speech Recognition (2019)0.00
• State-of-the-art Speech Recognition With Sequence-to-sequence Models (2017)21.01
• An Online Attention-based Model For Speech Recognition (2018)9.59
• Robust Sequence-to-sequence Acoustic Modeling With Stepwise Monotonic Attention For Neural TTS (2019)11.49
• Minimum Word Error Rate Training For Attention-based Sequence-to-sequence Models (2017)14.35
• Encoder-decoder With Focus-mechanism For Sequence Labelling Based Spoken Language Understanding (2016)0.00