Transfer Learning Of Language-independent End-to-end ASR With Language Model Fusion

Abstract

This work explores better adaptation methods to low-resource languages using an external language model (LM) under the framework of transfer learning. We first build a language-independent ASR system in a unified sequence-to-sequence (S2S) architecture with a shared vocabulary among all languages. During adaptation, we perform LM fusion transfer, where an external LM is integrated into the decoder network of the attention-based S2S model in the whole adaptation stage, to effectively incorporate linguistic context of the target language. We also investigate various seed models for transfer learning. Experimental evaluations using the IARPA BABEL data set show that LM fusion transfer improves performances on all target five languages compared with simple transfer learning when the external text data is available. Our final system drastically reduces the performance gap from the hybrid systems.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyinaguma2018transfer

Related papers

• Multilingual Sequence-to-sequence Speech Recognition: Architecture, Transfer Learning, And Language Modeling (2018)13.84
• Multilingual And Fully Non-autoregressive ASR With Large Language Model Fusion: A Comprehensive Study (2024)0.00
• Language Model Integration Based On Memory Control For Sequence To Sequence Speech Recognition (2018)2.26
• Adaptive Activation Network For Low Resource Multilingual Speech Recognition (2022)0.00
• Internal Language Model Estimation Based Adaptive Language Model Fusion For Domain Adaptation (2022)0.00
• Learning Cross-lingual Mappings For Data Augmentation To Improve Low-resource Speech Recognition (2023)0.00
• Parameter-efficient Adaptation Of Multilingual Multimodal Models For Low-resource ASR (2024)2.26
• An Analysis Of Incorporating An External Language Model Into A Sequence-to-sequence Model (2017)16.25