Multilingual Speech Recognition Using Knowledge Transfer Across Learning Processes

Abstract

Multilingual end-to-end(E2E) models have shown a great potential in the expansion of the language coverage in the realm of automatic speech recognition(ASR). In this paper, we aim to enhance the multilingual ASR performance in two ways, 1)studying the impact of feeding a one-hot vector identifying the language, 2)formulating the task with a meta-learning objective combined with self-supervised learning (SSL). We associate every language with a distinct task manifold and attempt to improve the performance by transferring knowledge across learning processes itself as compared to transferring through final model parameters. We employ this strategy on a dataset comprising of 6 languages for an in-domain ASR task, by minimizing an objective related to expected gradient path length. Experimental results reveal the best pre-training strategy resulting in 3.55% relative reduction in overall WER. A combination of LEAP and SSL yields 3.51% relative reduction in overall WER when using language ID

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylahiri2021multilingual

Related papers

• Improved Self-supervised Multilingual Speech Representation Learning Combined With Auxiliary Language Information (2022)0.00
• Improving Cross-lingual Transfer Learning For End-to-end Speech Recognition With Speech Translation (2020)9.92
• Large-scale Multilingual Speech Recognition With A Streaming End-to-end Model (2019)14.97
• Multilingual Speech Recognition With A Single End-to-end Model (2017)16.05
• Knowledge Transfer From Large-scale Pretrained Language Models To End-to-end Speech Recognizers (2022)9.41
• One-to-many Multilingual End-to-end Speech Translation (2019)9.23
• Language-universal Adapter Learning With Knowledge Distillation For End-to-end Multilingual Speech Recognition (2023)0.95
• Integrating Pre-trained Speech And Language Models For End-to-end Speech Recognition (2023)0.00