One-to-many Multilingual End-to-end Speech Translation

Abstract

Nowadays, training end-to-end neural models for spoken language translation (SLT) still has to confront with extreme data scarcity conditions. The existing SLT parallel corpora are indeed orders of magnitude smaller than those available for the closely related tasks of automatic speech recognition (ASR) and machine translation (MT), which usually comprise tens of millions of instances. To cope with data paucity, in this paper we explore the effectiveness of transfer learning in end-to-end SLT by presenting a multilingual approach to the task. Multilingual solutions are widely studied in MT and usually rely on ``\textit\{target forcing\}'', in which multilingual parallel data are combined to train a single model by prepending to the input sequences a language token that specifies the target language. However, when tested in speech translation, our experiments show that MT-like \textit\{target forcing\}, used as is, is not effective in discriminating among the target languages. Thus, we

Stats

• citations16
• S2 citations—
• github stars0
• HF likes0
• heat score9.23
• arxiv keydigangi2019one

Related papers

• Multilingual End-to-end Speech Translation (2019)0.00
• Enabling Zero-shot Multilingual Spoken Language Translation With Language-specific Encoders And Decoders (2020)2.26
• Leveraging Weakly Supervised Data To Improve End-to-end Speech-to-text Translation (2018)13.05
• Data Efficient Direct Speech-to-text Translation With Modality Agnostic Meta-learning (2019)0.00
• Joint Training And Decoding For Multilingual End-to-end Simultaneous Speech Translation (2025)0.95
• Making Llms Better Many-to-many Speech-to-text Translators With Curriculum Learning (2024)7.31
• Improving Cross-lingual Transfer Learning For End-to-end Speech Recognition With Speech Translation (2020)9.92
• Tackling Data Scarcity In Speech Translation Using Zero-shot Multilingual Machine Translation Techniques (2022)2.26