Self-training For End-to-end Speech Recognition

Abstract

We revisit self-training in the context of end-to-end speech recognition. We demonstrate that training with pseudo-labels can substantially improve the accuracy of a baseline model. Key to our approach are a strong baseline acoustic and language model used to generate the pseudo-labels, filtering mechanisms tailored to common errors from sequence-to-sequence models, and a novel ensemble approach to increase pseudo-label diversity. Experiments on the LibriSpeech corpus show that with an ensemble of four models and label filtering, self-training yields a 33.9% relative improvement in WER compared with a baseline trained on 100 hours of labelled data in the noisy speech setting. In the clean speech setting, self-training recovers 59.3% of the gap between the baseline and an oracle model, which is at least 93.8% relatively higher than what previous approaches can achieve.

Stats

• citations115
• S2 citations—
• github stars0
• HF likes0
• heat score15.48
• arxiv keykahn2019self

Related papers

• Self-training And Pre-training Are Complementary For Speech Recognition (2020)14.15
• Improving Pseudo-label Training For End-to-end Speech Recognition Using Gradient Mask (2021)5.84
• Joint Speech Transcription And Translation: Pseudo-labeling With Out-of-distribution Data (2022)0.00
• Censer: Curriculum Semi-supervised Learning For Speech Recognition Based On Self-supervised Pre-training (2022)4.52
• End-to-end ASR: From Supervised To Semi-supervised Learning With Modern Architectures (2019)0.00
• Wav2seq: Pre-training Speech-to-text Encoder-decoder Models Using Pseudo Languages (2022)10.48
• Semi-supervised Training With Pseudo-labeling For End-to-end Neural Diarization (2021)0.00
• Personalized Speech Enhancement Through Self-supervised Data Augmentation And Purification (2021)9.92