Accent-robust Automatic Speech Recognition Using Supervised And Unsupervised Wav2vec Embeddings

Abstract

Speech recognition models often obtain degraded performance when tested on speech with unseen accents. Domain-adversarial training (DAT) and multi-task learning (MTL) are two common approaches for building accent-robust ASR models. ASR models using accent embeddings is another approach for improving robustness to accents. In this study, we perform systematic comparisons of DAT and MTL approaches using a large volume of English accent corpus (4000 hours of US English speech and 1244 hours of 20 non-US-English accents speech). We explore embeddings trained under supervised and unsupervised settings: a separate embedding matrix trained using accent labels, and embeddings extracted from a fine-tuned wav2vec model. We find that our DAT model trained with supervised embeddings achieves the best performance overall and consistently provides benefits for all testing datasets, and our MTL model trained with wav2vec embeddings are helpful learning accent-invariant features and improving novel/un

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyli2021accent

Related papers

• Advanced Accent/dialect Identification And Accentedness Assessment With Multi-embedding Models And Automatic Speech Recognition (2023)7.16
• Best Of Both Worlds: Robust Accented Speech Recognition With Adversarial Transfer Learning (2021)9.23
• Accent Conversion In Text-to-speech Using Multi-level VAE And Adversarial Training (2024)5.84
• Synthetic Cross-accent Data Augmentation For Automatic Speech Recognition (2023)0.00
• English Accent Accuracy Analysis In A State-of-the-art Automatic Speech Recognition System (2021)0.00
• Layer-wise Fast Adaptation For End-to-end Multi-accent Speech Recognition (2022)9.76
• Whisper Turns Stronger: Augmenting Wav2vec 2.0 For Superior ASR In Low-resource Languages (2024)0.00
• Learning Asr-robust Contextualized Embeddings For Spoken Language Understanding (2019)12.02