Semantic Mask For Transformer Based End-to-end Speech Recognition

Abstract

Attention-based encoder-decoder model has achieved impressive results for both automatic speech recognition (ASR) and text-to-speech (TTS) tasks. This approach takes advantage of the memorization capacity of neural networks to learn the mapping from the input sequence to the output sequence from scratch, without the assumption of prior knowledge such as the alignments. However, this model is prone to overfitting, especially when the amount of training data is limited. Inspired by SpecAugment and BERT, in this paper, we propose a semantic mask based regularization for training such kind of end-to-end (E2E) model. The idea is to mask the input features corresponding to a particular output token, e.g., a word or a word-piece, in order to encourage the model to fill the token based on the contextual information. While this approach is applicable to the encoder-decoder framework with any type of neural network architecture, we study the transformer-based model for ASR in this work. We perfo

Stats

• citations17
• S2 citations—
• github stars0
• HF likes0
• heat score9.41
• arxiv keywang2019semantic

Related papers

• Effective Decoder Masking For Transformer Based End-to-end Speech Recognition (2020)0.00
• Simplified Self-attention For Transformer-based End-to-end Speech Recognition (2020)10.61
• Transmask: A Compact And Fast Speech Separation Model Based On Transformer (2021)8.82
• Telephonetic: Making Neural Language Models Robust To ASR And Semantic Noise (2019)0.00
• Improved Mask-ctc For Non-autoregressive End-to-end ASR (2020)11.76
• Improving Transformer-based Speech Recognition Using Unsupervised Pre-training (2019)0.00
• Improving Transformer-based Conversational ASR By Inter-sentential Attention Mechanism (2022)7.50
• S-transformer: Segment-transformer For Robust Neural Speech Synthesis (2020)0.00