Multilingual Sequence-to-sequence Speech Recognition: Architecture, Transfer Learning, And Language Modeling

Abstract

Sequence-to-sequence (seq2seq) approach for low-resource ASR is a relatively new direction in speech research. The approach benefits by performing model training without using lexicon and alignments. However, this poses a new problem of requiring more data compared to conventional DNN-HMM systems. In this work, we attempt to use data from 10 BABEL languages to build a multi-lingual seq2seq model as a prior model, and then port them towards 4 other BABEL languages using transfer learning approach. We also explore different architectures for improving the prior multilingual seq2seq model. The paper also discusses the effect of integrating a recurrent neural network language model (RNNLM) with a seq2seq model during decoding. Experimental results show that the transfer learning approach from the multilingual model shows substantial gains over monolingual models across all 4 BABEL languages. Incorporating an RNNLM also brings significant improvements in terms of %WER, and achieves recognit

Stats

• citations69
• S2 citations—
• github stars0
• HF likes0
• heat score13.84
• arxiv keycho2018multilingual

Related papers

• Analysis Of Multilingual Sequence-to-sequence Speech Recognition Systems (2018)0.00
• Transfer Learning Of Language-independent End-to-end ASR With Language Model Fusion (2018)0.00
• Sequence-based Multi-lingual Low Resource Speech Recognition (2018)12.40
• Adaptive Activation Network For Low Resource Multilingual Speech Recognition (2022)0.00
• Multilingual Speech Recognition With A Single End-to-end Model (2017)16.05
• Multilingual End-to-end Speech Recognition With A Single Transformer On Low-resource Languages (2018)0.00
• Towards Better Decoding And Language Model Integration In Sequence To Sequence Models (2016)15.67
• State-of-the-art Speech Recognition With Sequence-to-sequence Models (2017)21.01