Tagged End-to-end Simultaneous Speech Translation Training Using Simultaneous Interpretation Data

Abstract

Simultaneous speech translation (SimulST) translates partial speech inputs incrementally. Although the monotonic correspondence between input and output is preferable for smaller latency, it is not the case for distant language pairs such as English and Japanese. A prospective approach to this problem is to mimic simultaneous interpretation (SI) using SI data to train a SimulST model. However, the size of such SI data is limited, so the SI data should be used together with ordinary bilingual data whose translations are given in offline. In this paper, we propose an effective way to train a SimulST model using mixed data of SI and offline. The proposed method trains a single model using the mixed data with style tags that tell the model to generate SI- or offline-style outputs. Experiment results show improvements of BLEURT in different latency ranges, and our analyses revealed the proposed model generates SI-style outputs more than the baseline.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyko2023tagged

Related papers

• Efficient And Adaptive Simultaneous Speech Translation With Fully Unidirectional Architecture (2025)2.26
• Does Simultaneous Speech Translation Need Simultaneous Models? (2022)4.52
• End-to-end Simultaneous Speech Translation With Differentiable Segmentation (2023)7.16
• Simulsense: Sense-driven Interpreting For Efficient Simultaneous Speech Translation (2025)0.00
• Simuls2s-llm: Unlocking Simultaneous Inference Of Speech Llms For Speech-to-speech Translation (2025)3.58
• Joint Training And Decoding For Multilingual End-to-end Simultaneous Speech Translation (2025)0.95
• Visualization: The Missing Factor In Simultaneous Speech Translation (2021)0.00
• Streamspeech: Simultaneous Speech-to-speech Translation With Multi-task Learning (2024)7.81