Reducing Language Confusion For Code-switching Speech Recognition With Token-level Language Diarization
2022 Β· Hexin Liu, Haihua Xu, Leibny Paola Garcia, et al.
Abstract
Code-switching (CS) refers to the phenomenon that languages switch within a speech signal and leads to language confusion for automatic speech recognition (ASR). This paper aims to address language confusion for improving CS-ASR from two perspectives: incorporating and disentangling language information. We incorporate language information in the CS-ASR model by dynamically biasing the model with token-level language posteriors which are outputs of a sequence-to-sequence auxiliary language diarization module. In contrast, the disentangling process reduces the difference between languages via adversarial training so as to normalize two languages. We conduct the experiments on the SEAME dataset. Compared to the baseline model, both the joint optimization with LD and the language posterior bias achieve performance improvement. The comparison of the proposed methods indicates that incorporating language information is more effective than disentangling for reducing language confusion in CS
Authors
(none)
Tags
Stats
Related papers
- Aligning Speech To Languages To Enhance Code-switching Speech Recognition (2024)5.84
- Enhancing Code-switching Speech Recognition With Interactive Language Biases (2023)9.92
- Code-switching Speech Recognition Under The Lens: Model- And Data-centric Perspectives (2025)0.00
- Unified Model For Code-switching Speech Recognition And Language Identification Based On A Concatenated Tokenizer (2023)8.09
- Language-agnostic Code-switching In Sequence-to-sequence Speech Recognition (2022)0.00
- Integrating Knowledge In End-to-end Automatic Speech Recognition For Mandarin-english Code-switching (2021)5.24
- Code-switching Detection With Data-augmented Acoustic And Language Models (2018)3.58
- An Effective Mixture-of-experts Approach For Code-switching Speech Recognition Leveraging Encoder Disentanglement (2024)0.00