Automatic Pronunciation Assessment Using Self-supervised Speech Representation Learning
2022 · Eesung Kim, Jae-Jin Jeon, Hyeji Seo, et al.
Abstract
Self-supervised learning (SSL) approaches such as wav2vec 2.0 and HuBERT models have shown promising results in various downstream tasks in the speech community. In particular, speech representations learned by SSL models have been shown to be effective for encoding various speech-related characteristics. In this context, we propose a novel automatic pronunciation assessment method based on SSL models. First, the proposed method fine-tunes the pre-trained SSL models with connectionist temporal classification to adapt the English pronunciation of English-as-a-second-language (ESL) learners in a data environment. Then, the layer-wise contextual representations are extracted from all across the transformer layers of the SSL models. Finally, the automatic pronunciation score is estimated using bidirectional long short-term memory with the layer-wise contextual representations and the corresponding text. We show that the proposed SSL model-based methods outperform the baselines, in terms of
Authors
(none)
Tags
Stats
Related papers
- Efficient Infusion Of Self-supervised Representations In Automatic Speech Recognition (2024)0.00
- An Adapter Based Pre-training For Efficient And Scalable Self-supervised Speech Representation Learning (2021)8.35
- Unispeech-sat: Universal Speech Representation Learning With Speaker Aware Pre-training (2021)0.00
- Investigation Of Ensemble Features Of Self-supervised Pretrained Models For Automatic Speech Recognition (2022)9.41
- Non-contrastive Self-supervised Learning For Utterance-level Information Extraction From Speech (2022)9.59
- Deploying Self-supervised Learning In The Wild For Hybrid Automatic Speech Recognition (2022)0.00
- Self-supervised Learning For Speech Recognition With Intermediate Layer Supervision (2021)9.41
- Pushing The Limits Of Unsupervised Unit Discovery For SSL Speech Representation (2023)6.34