Sample, Translate, Recombine: Leveraging Audio Alignments For Data Augmentation In End-to-end Speech Translation
2022 · Tsz Kin Lam, Shigehiko Schamoni, Stefan Riezler
Abstract
End-to-end speech translation relies on data that pair source-language speech inputs with corresponding translations into a target language. Such data are notoriously scarce, making synthetic data augmentation by back-translation or knowledge distillation a necessary ingredient of end-to-end training. In this paper, we present a novel approach to data augmentation that leverages audio alignments, linguistic properties, and translation. First, we augment a transcription by sampling from a suffix memory that stores text and audio data. Second, we translate the augmented transcript. Finally, we recombine concatenated audio segments and the generated translation. Besides training an MT-system, we only use basic off-the-shelf components without fine-tuning. While having similar resource demands as knowledge distillation, adding our method delivers consistent improvements of up to 0.9 and 1.1 BLEU points on five language pairs on CoVoST 2 and on two language pairs on Europarl-ST, respectivel
Authors
(none)
Tags
Stats
Related papers
- Segaugment: Maximizing The Utility Of Speech Translation Data With Segmentation-based Augmentations (2022)0.00
- Skinaugment: Auto-encoding Speaker Conversions For Automatic Speech Translation (2020)7.16
- Harnessing Indirect Training Data For End-to-end Automatic Speech Translation: Tricks Of The Trade (2019)0.00
- Generating Synthetic Speech From Spokenvocab For Speech Translation (2022)5.38
- Alignatt: Using Attention-based Audio-translation Alignments As A Guide For Simultaneous Speech Translation (2023)3.58
- Leveraging Weakly Supervised Data To Improve End-to-end Speech-to-text Translation (2018)13.05
- Learning Cross-lingual Mappings For Data Augmentation To Improve Low-resource Speech Recognition (2023)0.00
- Pre-training End-to-end ASR Models With Augmented Speech Samples Queried By Text (2023)0.00