End-to-end Speech Translation With Knowledge Distillation

Abstract

End-to-end speech translation (ST), which directly translates from source language speech into target language text, has attracted intensive attentions in recent years. Compared to conventional pipeline systems, end-to-end ST models have advantages of lower latency, smaller model size and less error propagation. However, the combination of speech recognition and text translation in one model is more difficult than each of these two tasks. In this paper, we propose a knowledge distillation approach to improve ST model by transferring the knowledge from text translation model. Specifically, we first train a text translation model, regarded as a teacher model, and then ST model is trained to learn output probabilities from teacher model through knowledge distillation. Experiments on English- French Augmented LibriSpeech and English-Chinese TED corpus show that end-to-end ST is possible to implement on both similar and dissimilar language pairs. In addition, with the instruction of teacher

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyliu2019end

Related papers

• Decouple Non-parametric Knowledge Distillation For End-to-end Speech Translation (2023)0.00
• Source And Target Bidirectional Knowledge Distillation For End-to-end Speech Translation (2021)9.03
• Improving Cross-lingual Transfer Learning For End-to-end Speech Recognition With Speech Translation (2020)9.92
• Improving End-to-end Speech Translation By Imitation-based Knowledge Distillation With Synthetic Transcripts (2023)0.60
• Multilingual End-to-end Speech Translation (2019)0.00
• Two-stage Textual Knowledge Distillation For End-to-end Spoken Language Understanding (2020)9.41
• Data Efficient Direct Speech-to-text Translation With Modality Agnostic Meta-learning (2019)0.00
• Distilling Knowledge From Ensembles Of Acoustic Models For Joint Ctc-attention End-to-end Speech Recognition (2020)8.09