Language Adaptive Cross-lingual Speech Representation Learning With Sparse Sharing Sub-networks

Abstract

Unsupervised cross-lingual speech representation learning (XLSR) has recently shown promising results in speech recognition by leveraging vast amounts of unlabeled data across multiple languages. However, standard XLSR model suffers from language interference problem due to the lack of language specific modeling ability. In this work, we investigate language adaptive training on XLSR models. More importantly, we propose a novel language adaptive pre-training approach based on sparse sharing sub-networks. It makes room for language specific modeling by pruning out unimportant parameters for each language, without requiring any manually designed language specific component. After pruning, each language only maintains a sparse sub-network, while the sub-networks are partially shared with each other. Experimental results on a downstream multilingual speech recognition task show that our proposed method significantly outperforms baseline XLSR models on both high resource and low resource la

Stats

• citations12
• S2 citations—
• github stars0
• HF likes0
• heat score8.35
• arxiv keylu2022language

Related papers

• Unsupervised Cross-lingual Representation Learning For Speech Recognition (2020)18.91
• Improved Self-supervised Multilingual Speech Representation Learning Combined With Auxiliary Language Information (2022)0.00
• XLST: Cross-lingual Self-training To Learn Multilingual Representation For Low Resource Speech Recognition (2021)8.82
• Self-supervised Adaptive Pre-training Of Multilingual Speech Models For Language And Dialect Identification (2023)6.34
• Exploiting Cross-lingual Speaker And Phonetic Diversity For Unsupervised Subword Modeling (2019)6.77
• Towards Building Speech Large Language Models For Multitask Understanding In Low-resource Languages (2025)0.00
• Adaptive Activation Network For Low Resource Multilingual Speech Recognition (2022)0.00
• The Effectiveness Of Unsupervised Subword Modeling With Autoregressive And Cross-lingual Phone-aware Networks (2020)2.26