Multi-speaker Multi-style Text-to-speech Synthesis With Single-speaker Single-style Training Data Scenarios

Abstract

In the existing cross-speaker style transfer task, a source speaker with multi-style recordings is necessary to provide the style for a target speaker. However, it is hard for one speaker to express all expected styles. In this paper, a more general task, which is to produce expressive speech by combining any styles and timbres from a multi-speaker corpus in which each speaker has a unique style, is proposed. To realize this task, a novel method is proposed. This method is a Tacotron2-based framework but with a fine-grained text-based prosody predicting module and a speaker identity controller. Experiments demonstrate that the proposed method can successfully express a style of one speaker with the timber of another speaker bypassing the dependency on a single speaker's multi-style corpus. Moreover, the explicit prosody features used in the prosody predicting module can increase the diversity of synthetic speech by adjusting the value of prosody features.

Stats

• citations7
• S2 citations—
• github stars0
• HF likes0
• heat score6.77
• arxiv keyxie2021multi

Related papers

• Multi-reference Tacotron By Intercross Training For Style Disentangling,transfer And Control In Speech Synthesis (2019)0.00
• Cross-speaker Style Transfer With Prosody Bottleneck In Neural Speech Synthesis (2021)10.21
• Text-driven Emotional Style Control And Cross-speaker Style Transfer In Neural TTS (2022)7.81
• MM-TTS: Multi-modal Prompt Based Style Transfer For Expressive Text-to-speech Synthesis (2023)8.60
• Boosting Multi-speaker Expressive Speech Synthesis With Semi-supervised Contrastive Learning (2023)5.24
• Multi-speaker Multi-style Speech Synthesis With Timbre And Style Disentanglement (2022)6.77
• Expressive TTS Training With Frame And Style Reconstruction Loss (2020)12.74
• Exploring Synthetic Data For Cross-speaker Style Transfer In Style Representation Based TTS (2024)0.00