Direct Models For Simultaneous Translation And Automatic Subtitling: FBK@IWSLT2023

Abstract

This paper describes the FBK's participation in the Simultaneous Translation and Automatic Subtitling tracks of the IWSLT 2023 Evaluation Campaign. Our submission focused on the use of direct architectures to perform both tasks: for the simultaneous one, we leveraged the knowledge already acquired by offline-trained models and directly applied a policy to obtain the real-time inference; for the subtitling one, we adapted the direct ST model to produce well-formed subtitles and exploited the same architecture to produce timestamps needed for the subtitle synchronization with audiovisual content. Our English-German SimulST system shows a reduced computational-aware latency compared to the one achieved by the top-ranked systems in the 2021 and 2022 rounds of the task, with gains of up to 3.5 BLEU. Our automatic subtitling system outperforms the only existing solution based on a direct system by 3.7 and 1.7 SubER in English-German and English-Spanish respectively.

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keypapi2023direct

Related papers

• Efficient Yet Competitive Speech Translation: FBK@IWSLT2022 (2022)4.52
• Dealing With Training And Test Segmentation Mismatch: FBK@IWSLT2021 (2021)0.00
• Direct Speech Translation For Automatic Subtitling (2022)6.77
• Speech Translation With Foundation Models And Optimal Transport: UPC At IWSLT23 (2023)0.00
• FST: The FAIR Speech Translation System For The IWSLT21 Multilingual Shared Task (2021)0.00
• Kit's Low-resource Speech Translation Systems For IWSLT2025: System Enhancement With Synthetic Data And Model Regularization (2025)0.00
• End-to-end Speech Translation With Pre-trained Models And Adapters: UPC At IWSLT 2021 (2021)7.81
• Efficient And Adaptive Simultaneous Speech Translation With Fully Unidirectional Architecture (2025)2.26