Alternate Endings: Improving Prosody For Incremental Neural TTS With Predicted Future Text Input

Abstract

The prosody of a spoken word is determined by its surrounding context. In incremental text-to-speech synthesis, where the synthesizer produces an output before it has access to the complete input, the full context is often unknown which can result in a loss of naturalness in the synthesized speech. In this paper, we investigate whether the use of predicted future text can attenuate this loss. We compare several test conditions of next future word: (a) unknown (zero-word), (b) language model predicted, (c) randomly predicted and (d) ground-truth. We measure the prosodic features (pitch, energy and duration) and find that predicted text provides significant improvements over a zero-word lookahead, but only slight gains over random-word lookahead. We confirm these results with a perceptive test.

Stats

• citations6
• S2 citations—
• github stars0
• HF likes0
• heat score6.34
• arxiv keystephenson2021alternate

Related papers

• What The Future Brings: Investigating The Impact Of Lookahead For Incremental Neural TTS (2020)6.34
• Incremental Text-to-speech Synthesis Using Pseudo Lookahead With Large Pretrained Language Model (2020)8.82
• Applying Syntax\(\unicode{x2013}\)prosody Mapping Hypothesis And Prosodic Well-formedness Constraints To Neural Sequence-to-sequence Speech Synthesis (2022)0.00
• Controllable Neural Text-to-speech Synthesis Using Intuitive Prosodic Features (2020)11.76
• Prosody-controllable Spontaneous TTS With Neural Hmms (2022)8.09
• Dynamic Prosody Generation For Speech Synthesis Using Linguistics-driven Acoustic Embedding Selection (2019)7.81
• Hierarchical Prosody Modeling And Control In Non-autoregressive Parallel Neural TTS (2021)8.35
• Low-latency Incremental Text-to-speech Synthesis With Distilled Context Prediction Network (2021)6.34