R-BI: Regularized Batched Inputs Enhance Incremental Decoding Framework For Low-latency Simultaneous Speech Translation

Abstract

Incremental Decoding is an effective framework that enables the use of an offline model in a simultaneous setting without modifying the original model, making it suitable for Low-Latency Simultaneous Speech Translation. However, this framework may introduce errors when the system outputs from incomplete input. To reduce these output errors, several strategies such as Hold-\(n\), LA-\(n\), and SP-\(n\) can be employed, but the hyper-parameter \(n\) needs to be carefully selected for optimal performance. Moreover, these strategies are more suitable for end-to-end systems than cascade systems. In our paper, we propose a new adaptable and efficient policy named "Regularized Batched Inputs". Our method stands out by enhancing input diversity to mitigate output errors. We suggest particular regularization techniques for both end-to-end and cascade systems. We conducted experiments on IWSLT Simultaneous Speech Translation (SimulST) tasks, which demonstrate that our approach achieves low laten

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyguo2024r

Related papers

• Decision Attentive Regularization To Improve Simultaneous Speech Translation Systems (2021)0.00
• Efficient And Adaptive Simultaneous Speech Translation With Fully Unidirectional Architecture (2025)2.26
• Synchronous Speech Recognition And Speech-to-text Translation With Interactive Decoding (2019)10.48
• Impact Of Encoding And Segmentation Strategies On End-to-end Simultaneous Speech Translation (2021)4.52
• Forward-backward Decoding For Regularizing End-to-end TTS (2019)6.77
• Improving Stability In Simultaneous Speech Translation: A Revision-controllable Decoding Approach (2023)2.26
• Exploring Continuous Integrate-and-fire For Adaptive Simultaneous Speech Translation (2022)4.52
• Learning When To Speak: Latency And Quality Trade-offs For Simultaneous Speech-to-speech Translation With Offline Models (2023)0.00