Improving End-to-end Modeling For Mispronunciation Detection With Effective Augmentation Mechanisms
2021 · Tien-Hong Lo, Yao-Ting Sung, Berlin Chen
Abstract
Recently, end-to-end (E2E) models, which allow to take spectral vector sequences of L2 (second-language) learners' utterances as input and produce the corresponding phone-level sequences as output, have attracted much research attention in developing mispronunciation detection (MD) systems. However, due to the lack of sufficient labeled speech data of L2 speakers for model estimation, E2E MD models are prone to overfitting in relation to conventional ones that are built on DNN-HMM acoustic models. To alleviate this critical issue, we in this paper propose two modeling strategies to enhance the discrimination capability of E2E MD models, each of which can implicitly leverage the phonetic and phonological traits encoded in a pretrained acoustic model and contained within reference transcripts of the training data, respectively. The first one is input augmentation, which aims to distill knowledge about phonetic discrimination from a DNN-HMM acoustic model. The second one is label augmenta
Authors
(none)
Tags
Stats
Related papers
- A Full Text-dependent End To End Mispronunciation Detection And Diagnosis With Easy Data Augmentation Techniques (2021)0.00
- Improving Mispronunciation Detection With Wav2vec2-based Momentum Pseudo-labeling For Accentedness And Intelligibility Assessment (2022)7.16
- Speechblender: Speech Augmentation Framework For Mispronunciation Data Generation (2022)2.26
- Hmm-based Data Augmentation For E2E Systems For Building Conversational Speech Synthesis Systems (2022)0.00
- Data Augmentation Methods For End-to-end Speech Recognition On Distant-talk Scenarios (2021)6.34
- Multi-view Multi-task Representation Learning For Mispronunciation Detection (2023)0.00
- Integrating Pre-trained Speech And Language Models For End-to-end Speech Recognition (2023)0.00
- Telephonetic: Making Neural Language Models Robust To ASR And Semantic Noise (2019)0.00