Language-agnostic Code-switching In Sequence-to-sequence Speech Recognition

Abstract

Code-Switching (CS) is referred to the phenomenon of alternately using words and phrases from different languages. While today's neural end-to-end (E2E) models deliver state-of-the-art performances on the task of automatic speech recognition (ASR) it is commonly known that these systems are very data-intensive. However, there is only a few transcribed and aligned CS speech available. To overcome this problem and train multilingual systems which can transcribe CS speech, we propose a simple yet effective data augmentation in which audio and corresponding labels of different source languages are concatenated. By using this training data, our E2E model improves on transcribing CS speech. It also surpasses monolingual models on monolingual tests. The results show that this augmentation technique can even improve the model's performance on inter-sentential language switches not seen during training by 5,03% WER.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyugan2022language

Related papers

• On The End-to-end Solution To Mandarin-english Code-switching Speech Recognition (2018)12.10
• Code-switching Without Switching: Language Agnostic End-to-end Speech Translation (2022)0.00
• Code-switching Speech Recognition Under The Lens: Model- And Data-centric Perspectives (2025)0.00
• Integrating Knowledge In End-to-end Automatic Speech Recognition For Mandarin-english Code-switching (2021)5.24
• End-to-end Code-switching ASR For Low-resourced Language Pairs (2019)9.76
• Unified Model For Code-switching Speech Recognition And Language Identification Based On A Concatenated Tokenizer (2023)8.09
• Textual Data Augmentation For Arabic-english Code-switching Speech Recognition (2022)6.77
• Data Augmentation For End-to-end Code-switching Speech Recognition (2020)9.92