Bootstrap An End-to-end ASR System By Multilingual Training, Transfer Learning, Text-to-text Mapping And Synthetic Audio

Abstract

Bootstrapping speech recognition on limited data resources has been an area of active research for long. The recent transition to all-neural models and end-to-end (E2E) training brought along particular challenges as these models are known to be data hungry, but also came with opportunities around language-agnostic representations derived from multilingual data as well as shared word-piece output representations across languages that share script and roots. We investigate here the effectiveness of different strategies to bootstrap an RNN-Transducer (RNN-T) based automatic speech recognition (ASR) system in the low resource regime, while exploiting the abundant resources available in other languages as well as the synthetic audio from a text-to-speech (TTS) engine. Our experiments demonstrate that transfer learning from a multilingual model, using a post-ASR text-to-text mapping and synthetic audio deliver additive improvements, allowing us to bootstrap a model for a new language with a

Stats

• citations4
• S2 citations—
• github stars0
• HF likes0
• heat score5.24
• arxiv keygiollo2020bootstrap

Related papers

• Language Model Bootstrapping Using Neural Machine Translation For Conversational Speech Recognition (2019)5.24
• ASR Data Augmentation In Low-resource Settings Using Cross-lingual Multi-speaker TTS And Cross-lingual Voice Conversion (2022)6.77
• End-to-end Text-to-speech For Low-resource Languages By Cross-lingual Transfer Learning (2019)0.00
• Rapid Speaker Adaptation In Low Resource Text To Speech Systems Using Synthetic Data And Transfer Learning (2023)0.00
• Learning Cross-lingual Mappings For Data Augmentation To Improve Low-resource Speech Recognition (2023)0.00
• Adaptive Activation Network For Low Resource Multilingual Speech Recognition (2022)0.00
• Integrating Text Inputs For Training And Adapting RNN Transducer ASR Models (2022)9.59
• Almost Unsupervised Text To Speech And Automatic Speech Recognition (2019)0.00