E2e-based Multi-task Learning Approach To Joint Speech And Accent Recognition

Abstract

In this paper, we propose a single multi-task learning framework to perform End-to-End (E2E) speech recognition (ASR) and accent recognition (AR) simultaneously. The proposed framework is not only more compact but can also yield comparable or even better results than standalone systems. Specifically, we found that the overall performance is predominantly determined by the ASR task, and the E2E-based ASR pretraining is essential to achieve improved performance, particularly for the AR task. Additionally, we conduct several analyses of the proposed method. First, though the objective loss for the AR task is much smaller compared with its counterpart of ASR task, a smaller weighting factor with the AR task in the joint objective function is necessary to yield better results for each task. Second, we found that sharing only a few layers of the encoder yields better AR results than sharing the overall encoder. Experimentally, the proposed method produces WER results close to the best standa

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhang2021e2e

Related papers

• Decoupling And Interacting Multi-task Learning Network For Joint Speech And Accent Recognition (2023)9.03
• Unified End-to-end Speech Recognition And Endpointing For Fast And Efficient Speech Systems (2022)5.24
• Multi-encoder Multi-resolution Framework For End-to-end Speech Recognition (2018)0.00
• Joint Ctc-attention Based End-to-end Speech Recognition Using Multi-task Learning (2016)20.43
• Multi-stream End-to-end Speech Recognition (2019)8.35
• Multilingual Approach To Joint Speech And Accent Recognition With DNN-HMM Framework (2020)0.00
• Integrating Pre-trained Speech And Language Models For End-to-end Speech Recognition (2023)0.00
• Attention-augmented End-to-end Multi-task Learning For Emotion Prediction From Speech (2019)13.50