Unified Model For Code-switching Speech Recognition And Language Identification Based On A Concatenated Tokenizer

Abstract

Code-Switching (CS) multilingual Automatic Speech Recognition (ASR) models can transcribe speech containing two or more alternating languages during a conversation. This paper proposes (1) a new method for creating code-switching ASR datasets from purely monolingual data sources, and (2) a novel Concatenated Tokenizer that enables ASR models to generate language ID for each emitted text token while reusing existing monolingual tokenizers. The efficacy of these approaches for building CS ASR models is demonstrated for two language pairs, English-Hindi and English-Spanish, where we achieve new state-of-the-art results on the Miami Bangor CS evaluation corpus. In addition to competitive ASR performance, the proposed Concatenated Tokenizer models are highly effective for spoken language identification, achieving 98%+ accuracy on the out-of-distribution FLEURS dataset.

Stats

• citations11
• S2 citations—
• github stars0
• HF likes0
• heat score8.09
• arxiv keydhawan2023unified

Related papers

• Language-agnostic Code-switching In Sequence-to-sequence Speech Recognition (2022)0.00
• Code-switching Speech Recognition Under The Lens: Model- And Data-centric Perspectives (2025)0.00
• Integrating Knowledge In End-to-end Automatic Speech Recognition For Mandarin-english Code-switching (2021)5.24
• Balanced End-to-end Monolingual Pre-training For Low-resourced Indic Languages Code-switching Speech Recognition (2021)0.00
• Language Modeling For Code-switching: Evaluation, Integration Of Monolingual Data, And Discriminative Training (2018)5.24
• Code-switching Detection With Data-augmented Acoustic And Language Models (2018)3.58
• Reducing Language Confusion For Code-switching Speech Recognition With Token-level Language Diarization (2022)10.07
• Transformer-transducers For Code-switched Speech Recognition (2020)10.97