Incremental Text-to-speech Synthesis Using Pseudo Lookahead With Large Pretrained Language Model

Abstract

This letter presents an incremental text-to-speech (TTS) method that performs synthesis in small linguistic units while maintaining the naturalness of output speech. Incremental TTS is generally subject to a trade-off between latency and synthetic speech quality. It is challenging to produce high-quality speech with a low-latency setup that does not make much use of an unobserved future sentence (hereafter, "lookahead"). To resolve this issue, we propose an incremental TTS method that uses a pseudo lookahead generated with a language model to take the future contextual information into account without increasing latency. Our method can be regarded as imitating a human's incremental reading and uses pretrained GPT2, which accounts for the large-scale linguistic knowledge, for the lookahead generation. Evaluation results show that our method 1) achieves higher speech quality than the method taking only observed information into account and 2) achieves a speech quality equivalent to waiti

Stats

• citations14
• S2 citations—
• github stars0
• HF likes0
• heat score8.82
• arxiv keysaeki2020incremental

Related papers

• What The Future Brings: Investigating The Impact Of Lookahead For Incremental Neural TTS (2020)6.34
• Low-latency Incremental Text-to-speech Synthesis With Distilled Context Prediction Network (2021)6.34
• Incremental Text-to-speech Synthesis With Prefix-to-prefix Framework (2019)7.81
• Incremental Text To Speech For Neural Sequence-to-sequence Models Using Reinforcement Learning (2020)5.24
• Alternate Endings: Improving Prosody For Incremental Neural TTS With Predicted Future Text Input (2021)6.34
• From Start To Finish: Latency Reduction Strategies For Incremental Speech Synthesis In Simultaneous Speech-to-speech Translation (2021)2.26
• Generative Pre-trained Speech Language Model With Efficient Hierarchical Transformer (2024)5.96
• Evaluating Text-to-speech Synthesis From A Large Discrete Token-based Speech Language Model (2024)0.00