Pre-training End-to-end ASR Models With Augmented Speech Samples Queried By Text

Abstract

In end-to-end automatic speech recognition system, one of the difficulties for language expansion is the limited paired speech and text training data. In this paper, we propose a novel method to generate augmented samples with unpaired speech feature segments and text data for model pre-training, which has the advantage of low cost without using additional speech data. When mixing 20,000 hours augmented speech data generated by our method with 12,500 hours original transcribed speech data for Italian Transformer transducer model pre-training, we achieve 8.7% relative word error rate reduction. The pre-trained model achieves similar performance as the model pre-trained with multilingual transcribed 75,000 hours raw speech data. When merging the augmented speech data with the multilingual data to pre-train a new model, we achieve even more relative word error rate reduction of 12.2% over the baseline, which further verifies the effectiveness of our method for speech data augmentation.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keysun2023pre

Related papers

• Pretraining By Backtranslation For End-to-end ASR In Low-resource Settings (2018)0.00
• Back-translation-style Data Augmentation For End-to-end ASR (2018)13.11
• Analyzing ASR Pretraining For Low-resource Speech-to-text Translation (2019)10.07
• You Do Not Need More Data: Improving End-to-end Speech Recognition By Text-to-speech Data Augmentation (2020)11.49
• ASR Data Augmentation In Low-resource Settings Using Cross-lingual Multi-speaker TTS And Cross-lingual Voice Conversion (2022)6.77
• Harnessing Indirect Training Data For End-to-end Automatic Speech Translation: Tricks Of The Trade (2019)0.00
• Data Augmentation For End-to-end Code-switching Speech Recognition (2020)9.92
• Sample, Translate, Recombine: Leveraging Audio Alignments For Data Augmentation In End-to-end Speech Translation (2022)7.81