Multilingual Training And Cross-lingual Adaptation On Ctc-based Acoustic Model

Abstract

Multilingual models for Automatic Speech Recognition (ASR) are attractive as they have been shown to benefit from more training data, and better lend themselves to adaptation to under-resourced languages. However, initialisation from monolingual context-dependent models leads to an explosion of context-dependent states. Connectionist Temporal Classification (CTC) is a potential solution to this as it performs well with monophone labels. We investigate multilingual CTC in the context of adaptation and regularisation techniques that have been shown to be beneficial in more conventional contexts. The multilingual model is trained to model a universal International Phonetic Alphabet (IPA)-based phone set using the CTC loss function. Learning Hidden Unit Contribution (LHUC) is investigated to perform language adaptive training. In addition, dropout during cross-lingual adaptation is also studied and tested in order to mitigate the overfitting problem. Experiments show that the performan

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keytong2017multilingual

Related papers

• Multiple-hypothesis Ctc-based Semi-supervised Adaptation Of End-to-end Speech Recognition (2021)5.84
• Multilingual Adaptation Of RNN Based ASR Systems (2017)7.50
• Speaker Adaptation For End-to-end CTC Models (2019)8.60
• Continual Learning For Monolingual End-to-end Automatic Speech Recognition (2021)7.16
• Multilingual Contextual Adapters To Improve Custom Word Recognition In Low-resource Languages (2023)4.52
• Advancing Multi-accented LSTM-CTC Speech Recognition Using A Domain Specific Student-teacher Learning Paradigm (2018)7.81
• Hierarchical Multitask Learning For Ctc-based Speech Recognition (2018)0.00
• Reducing Spelling Inconsistencies In Code-switching ASR Using Contextualized CTC Loss (2020)4.52