End-to-end ASR: From Supervised To Semi-supervised Learning With Modern Architectures

Abstract

We study pseudo-labeling for the semi-supervised training of ResNet, Time-Depth Separable ConvNets, and Transformers for speech recognition, with either CTC or Seq2Seq loss functions. We perform experiments on the standard LibriSpeech dataset, and leverage additional unlabeled data from LibriVox through pseudo-labeling. We show that while Transformer-based acoustic models have superior performance with the supervised dataset alone, semi-supervision improves all models across architectures and loss functions and bridges much of the performance gaps between them. In doing so, we reach a new state-of-the-art for end-to-end acoustic models decoded with an external language model in the standard supervised learning setting, and a new absolute state-of-the-art with semi-supervised training. Finally, we study the effect of leveraging different amounts of unlabeled audio, propose several ways of evaluating the characteristics of unlabeled audio which improve acoustic modeling, and show that ac

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keysynnaeve2019end

Related papers

• Deep Contextualized Acoustic Representations For Semi-supervised Speech Recognition (2019)14.62
• Semi-supervised Sequence-to-sequence ASR Using Unpaired Speech And Text (2019)0.00
• Improving Hybrid Ctc/attention End-to-end Speech Recognition With Pretrained Acoustic And Language Model (2021)8.82
• Improving Pseudo-label Training For End-to-end Speech Recognition Using Gradient Mask (2021)5.84
• End-to-end Rich Transcription-style Automatic Speech Recognition With Semi-supervised Learning (2021)4.52
• Bigssl: Exploring The Frontier Of Large-scale Semi-supervised Learning For Automatic Speech Recognition (2021)15.73
• Self-training For End-to-end Speech Recognition (2019)15.48
• Censer: Curriculum Semi-supervised Learning For Speech Recognition Based On Self-supervised Pre-training (2022)4.52