Towards End-to-end Prosody Transfer For Expressive Speech Synthesis With Tacotron

Abstract

We present an extension to the Tacotron speech synthesis architecture that learns a latent embedding space of prosody, derived from a reference acoustic representation containing the desired prosody. We show that conditioning Tacotron on this learned embedding space results in synthesized audio that matches the prosody of the reference signal with fine time detail even when the reference and synthesis speakers are different. Additionally, we show that a reference prosody embedding can be used to synthesize text that is different from that of the reference utterance. We define several quantitative and subjective metrics for evaluating prosody transfer, and report results with accompanying audio samples from single-speaker and 44-speaker Tacotron models on a prosody transfer task.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyskerryryan2018towards

Related papers

• Prosody Transfer In Neural Text To Speech Using Global Pitch And Loudness Features (2019)0.00
• Tacotron: Towards End-to-end Speech Synthesis (2017)0.00
• Controllable Emotion Transfer For End-to-end Speech Synthesis (2020)13.05
• Modeling Prosodic Phrasing With Multi-task Learning In Tacotron-based TTS (2020)9.41
• Using Ipa-based Tacotron For Data Efficient Cross-lingual Speaker Adaptation And Pronunciation Enhancement (2020)0.00
• Controllable Neural Text-to-speech Synthesis Using Intuitive Prosodic Features (2020)11.76
• Multi-speaker Multi-style Text-to-speech Synthesis With Single-speaker Single-style Training Data Scenarios (2021)6.77
• Cross-speaker Emotion Disentangling And Transfer For End-to-end Speech Synthesis (2021)12.61