Reducing Spelling Inconsistencies In Code-switching ASR Using Contextualized CTC Loss

Abstract

Code-Switching (CS) remains a challenge for Automatic Speech Recognition (ASR), especially character-based models. With the combined choice of characters from multiple languages, the outcome from character-based models suffers from phoneme duplication, resulting in language-inconsistent spellings. We propose Contextualized Connectionist Temporal Classification (CCTC) loss to encourage spelling consistencies of a character-based non-autoregressive ASR which allows for faster inference. The CCTC loss conditions the main prediction on the predicted contexts to ensure language consistency in the spellings. In contrast to existing CTC-based approaches, CCTC loss does not require frame-level alignments, since the context ground truth is obtained from the model's estimated path. Compared to the same model trained with regular CTC loss, our method consistently improved the ASR performance on both CS and monolingual corpora.

Stats

• citations3
• S2 citations—
• github stars0
• HF likes0
• heat score4.52
• arxiv keynaowarat2020reducing

Related papers

• Multilingual Training And Cross-lingual Adaptation On Ctc-based Acoustic Model (2017)0.00
• Leveraging Language ID To Calculate Intermediate CTC Loss For Enhanced Code-switching Speech Recognition (2023)0.00
• Code-switching Speech Recognition Under The Lens: Model- And Data-centric Perspectives (2025)0.00
• Improving Zero-shot Chinese-english Code-switching ASR With Knn-ctc And Gated Monolingual Datastores (2024)0.00
• Multilingual Contextual Adapters To Improve Custom Word Recognition In Low-resource Languages (2023)4.52
• Fast Context-biasing For CTC And Transducer ASR Models With Ctc-based Word Spotter (2024)2.26
• CR-CTC: Consistency Regularization On CTC For Improved Speech Recognition (2024)6.30
• Towards End-to-end Code-switching Speech Recognition (2018)0.00