Integrating Knowledge In End-to-end Automatic Speech Recognition For Mandarin-english Code-switching

Abstract

Code-Switching (CS) is a common linguistic phenomenon in multilingual communities that consists of switching between languages while speaking. This paper presents our investigations on end-to-end speech recognition for Mandarin-English CS speech. We analyse different CS specific issues such as the properties mismatches between languages in a CS language pair, the unpredictable nature of switching points, and the data scarcity problem. We exploit and improve the state-of-the-art end-to-end system by merging nonlinguistic symbols, by integrating language identification using hierarchical softmax, by modeling sub-word units, by artificially lowering the speaking rate, and by augmenting data using speed perturbed technique and several monolingual datasets to improve the final performance not only on CS speech but also on monolingual benchmarks in order to make the system more applicable on real life settings. Finally, we explore the effect of different language model integration methods on

Stats

• citations4
• S2 citations—
• github stars0
• HF likes0
• heat score5.24
• arxiv keyli2021integrating

Related papers

• Towards End-to-end Code-switching Speech Recognition (2018)0.00
• On The End-to-end Solution To Mandarin-english Code-switching Speech Recognition (2018)12.10
• Language-agnostic Code-switching In Sequence-to-sequence Speech Recognition (2022)0.00
• The ASRU 2019 Mandarin-english Code-switching Speech Recognition Challenge: Open Datasets, Tracks, Methods And Results (2020)0.00
• Exploring Retraining-free Speech Recognition For Intra-sentential Code-switching (2021)5.84
• Code-switching Speech Recognition Under The Lens: Model- And Data-centric Perspectives (2025)0.00
• End-to-end Code-switching ASR For Low-resourced Language Pairs (2019)9.76
• Unified Model For Code-switching Speech Recognition And Language Identification Based On A Concatenated Tokenizer (2023)8.09