Improving Pseudo-label Training For End-to-end Speech Recognition Using Gradient Mask

Abstract

In the recent trend of semi-supervised speech recognition, both self-supervised representation learning and pseudo-labeling have shown promising results. In this paper, we propose a novel approach to combine their ideas for end-to-end speech recognition model. Without any extra loss function, we utilize the Gradient Mask to optimize the model when training on pseudo-label. This method forces the speech recognition model to predict from the masked input to learn strong acoustic representation and make training robust to label noise. In our semi-supervised experiments, the method can improve the model performance when training on pseudo-label and our method achieved competitive results comparing with other semi-supervised approaches on the Librispeech 100 hours experiments.

Stats

• citations5
• S2 citations—
• github stars0
• HF likes0
• heat score5.84
• arxiv keyling2021improving

Related papers

• Self-training For End-to-end Speech Recognition (2019)15.48
• Alternative Pseudo-labeling For Semi-supervised Automatic Speech Recognition (2023)10.48
• Self-supervised Speaker Recognition With Loss-gated Learning (2021)16.93
• End-to-end ASR: From Supervised To Semi-supervised Learning With Modern Architectures (2019)0.00
• Joint Speech Transcription And Translation: Pseudo-labeling With Out-of-distribution Data (2022)0.00
• Advancing Momentum Pseudo-labeling With Conformer And Initialization Strategy (2021)6.34
• Semi-supervised Training With Pseudo-labeling For End-to-end Neural Diarization (2021)0.00
• Self-training And Pre-training Are Complementary For Speech Recognition (2020)14.15