Multi-accent Adaptation Based On Gate Mechanism

Abstract

When only a limited amount of accented speech data is available, to promote multi-accent speech recognition performance, the conventional approach is accent-specific adaptation, which adapts the baseline model to multiple target accents independently. To simplify the adaptation procedure, we explore adapting the baseline model to multiple target accents simultaneously with multi-accent mixed data. Thus, we propose using accent-specific top layer with gate mechanism (AST-G) to realize multi-accent adaptation. Compared with the baseline model and accent-specific adaptation, AST-G achieves 9.8% and 1.9% average relative WER reduction respectively. However, in real-world applications, we can't obtain the accent category label for inference in advance. Therefore, we apply using an accent classifier to predict the accent label. To jointly train the acoustic model and the accent classifier, we propose the multi-task learning with gate mechanism (MTL-G). As the accent label prediction could be

Stats

• citations12
• S2 citations—
• github stars0
• HF likes0
• heat score8.35
• arxiv keyzhu2020multi

Related papers

• Layer-wise Fast Adaptation For End-to-end Multi-accent Speech Recognition (2022)9.76
• Attention-based Gated Scaling Adaptative Acoustic Model For Ctc-based Speech Recognition (2019)0.00
• A Highly Adaptive Acoustic Model For Accurate Multi-dialect Speech Recognition (2022)10.85
• Multi-scale Accent Modeling And Disentangling For Multi-speaker Multi-accent Text-to-speech Synthesis (2024)2.26
• Accent-robust Automatic Speech Recognition Using Supervised And Unsupervised Wav2vec Embeddings (2021)0.00
• Improving Accent Conversion With Reference Encoder And End-to-end Text-to-speech (2020)0.00
• E2e-based Multi-task Learning Approach To Joint Speech And Accent Recognition (2021)0.00
• Synthetic Cross-accent Data Augmentation For Automatic Speech Recognition (2023)0.00