From Start To Finish: Latency Reduction Strategies For Incremental Speech Synthesis In Simultaneous Speech-to-speech Translation
2021 · Danni Liu, Changhan Wang, Hongyu Gong, et al.
Abstract
Speech-to-speech translation (S2ST) converts input speech to speech in another language. A challenge of delivering S2ST in real time is the accumulated delay between the translation and speech synthesis modules. While recently incremental text-to-speech (iTTS) models have shown large quality improvements, they typically require additional future text inputs to reach optimal performance. In this work, we minimize the initial waiting time of iTTS by adapting the upstream speech translator to generate high-quality pseudo lookahead for the speech synthesizer. After mitigating the initial delay, we demonstrate that the duration of synthesized speech also plays a crucial role on latency. We formalize this as a latency metric and then present a simple yet effective duration-scaling approach for latency reduction. Our approaches consistently reduce latency by 0.2-0.5 second without sacrificing speech translation quality.
Authors
(none)
Tags
Stats
Related papers
- Learning When To Speak: Latency And Quality Trade-offs For Simultaneous Speech-to-speech Translation With Offline Models (2023)0.00
- What Does It Take To Get State Of The Art In Simultaneous Speech-to-speech Translation? (2024)0.00
- CA*: Addressing Evaluation Pitfalls In Computation-aware Latency For Simultaneous Speech Translation (2024)0.00
- Efficient And Adaptive Simultaneous Speech Translation With Fully Unidirectional Architecture (2025)2.26
- Synchronous Speech Recognition And Speech-to-text Translation With Interactive Decoding (2019)10.48
- Simuls2s-llm: Unlocking Simultaneous Inference Of Speech Llms For Speech-to-speech Translation (2025)3.58
- Visualization: The Missing Factor In Simultaneous Speech Translation (2021)0.00
- Isochrony-controlled Speech-to-text Translation: A Study On Translating From Sino-tibetan To Indo-european Languages (2024)0.00