Investigating The Impact Of Cross-lingual Acoustic-phonetic Similarities On Multilingual Speech Recognition

Abstract

Multilingual automatic speech recognition (ASR) systems mostly benefit low resource languages but suffer degradation in performance across several languages relative to their monolingual counterparts. Limited studies have focused on understanding the languages behaviour in the multilingual speech recognition setups. In this paper, a novel data-driven approach is proposed to investigate the cross-lingual acoustic-phonetic similarities. This technique measures the similarities between posterior distributions from various monolingual acoustic models against a target speech signal. Deep neural networks are trained as mapping networks to transform the distributions from different acoustic models into a directly comparable form. The analysis observes that the languages closeness can not be truly estimated by the volume of overlapping phonemes set. Entropy analysis of the proposed mapping networks exhibits that a language with lesser overlap can be more amenable to cross-lingual transfer, and

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keyfarooq2022investigating

Related papers

• Learning Cross-lingual Mappings For Data Augmentation To Improve Low-resource Speech Recognition (2023)0.00
• Multi-staged Cross-lingual Acoustic Model Adaption For Robust Speech Recognition In Real-world Applications -- A Case Study On German Oral History Interviews (2020)0.00
• Massively Multilingual Adversarial Speech Recognition (2019)11.93
• Exploiting Cross-lingual Speaker And Phonetic Diversity For Unsupervised Subword Modeling (2019)6.77
• Adaptive Activation Network For Low Resource Multilingual Speech Recognition (2022)0.00
• Cross-lingual Low Resource Speaker Adaptation Using Phonological Features (2021)5.24
• A Two-stage Transliteration Approach To Improve Performance Of A Multilingual ASR (2024)0.00
• How Familiar Does That Sound? Cross-lingual Representational Similarity Analysis Of Acoustic Word Embeddings (2021)0.00