Leveraging Language ID To Calculate Intermediate CTC Loss For Enhanced Code-switching Speech Recognition

Abstract

In recent years, end-to-end speech recognition has emerged as a technology that integrates the acoustic, pronunciation dictionary, and language model components of the traditional Automatic Speech Recognition model. It is possible to achieve human-like recognition without the need to build a pronunciation dictionary in advance. However, due to the relative scarcity of training data on code-switching, the performance of ASR models tends to degrade drastically when encountering this phenomenon. Most past studies have simplified the learning complexity of the model by splitting the code-switching task into multiple tasks dealing with a single language and then learning the domain-specific knowledge of each language separately. Therefore, in this paper, we attempt to introduce language identification information into the middle layer of the ASR model's encoder. We aim to generate acoustic features that imply language distinctions in a more implicit way, reducing the model's confusion when

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyyang2023leveraging

Related papers

• Towards End-to-end Code-switching Speech Recognition (2018)0.00
• Rnn-transducer With Language Bias For End-to-end Mandarin-english Code-switching Speech Recognition (2020)8.09
• Transformer-transducers For Code-switched Speech Recognition (2020)10.97
• Reducing Spelling Inconsistencies In Code-switching ASR Using Contextualized CTC Loss (2020)4.52
• Integrating Knowledge In End-to-end Automatic Speech Recognition For Mandarin-english Code-switching (2021)5.24
• Unified Model For Code-switching Speech Recognition And Language Identification Based On A Concatenated Tokenizer (2023)8.09
• Aligning Speech To Languages To Enhance Code-switching Speech Recognition (2024)5.84
• An Effective Mixture-of-experts Approach For Code-switching Speech Recognition Leveraging Encoder Disentanglement (2024)0.00