Prosodic Clustering For Phoneme-level Prosody Control In End-to-end Speech Synthesis

Abstract

This paper presents a method for controlling the prosody at the phoneme level in an autoregressive attention-based text-to-speech system. Instead of learning latent prosodic features with a variational framework as is commonly done, we directly extract phoneme-level F0 and duration features from the speech data in the training set. Each prosodic feature is discretized using unsupervised clustering in order to produce a sequence of prosodic labels for each utterance. This sequence is used in parallel to the phoneme sequence in order to condition the decoder with the utilization of a prosodic encoder and a corresponding attention module. Experimental results show that the proposed method retains the high quality of generated speech, while allowing phoneme-level control of F0 and duration. By replacing the F0 cluster centroids with musical notes, the model can also provide control over the note and octave within the range of the speaker.

Stats

• citations5
• S2 citations—
• github stars0
• HF likes0
• heat score5.84
• arxiv keyvioni2021prosodic

Related papers

• Controllable Speech Synthesis By Learning Discrete Phoneme-level Prosodic Representations (2022)6.34
• Improved Prosodic Clustering For Multispeaker And Speaker-independent Phoneme-level Prosody Control (2021)0.00
• Perception Of Prosodic Variation For Speech Synthesis Using An Unsupervised Discrete Representation Of F0 (2020)7.81
• Hierarchical Prosody Modeling And Control In Non-autoregressive Parallel Neural TTS (2021)8.35
• Unsupervised Word-level Prosody Tagging For Controllable Speech Synthesis (2022)7.16
• Hierarchical Prosody Modeling For Non-autoregressive Speech Synthesis (2020)10.07
• Robust And Fine-grained Prosody Control Of End-to-end Speech Synthesis (2018)14.31
• Emphasis Control For Parallel Neural TTS (2021)6.77