Textless Speech Emotion Conversion Using Discrete And Decomposed Representations
2021 · Felix Kreuk, Adam Polyak, Jade Copet, et al.
Abstract
Speech emotion conversion is the task of modifying the perceived emotion of a speech utterance while preserving the lexical content and speaker identity. In this study, we cast the problem of emotion conversion as a spoken language translation task. We use a decomposition of the speech signal into discrete learned representations, consisting of phonetic-content units, prosodic features, speaker, and emotion. First, we modify the speech content by translating the phonetic-content units to a target emotion, and then predict the prosodic features based on these units. Finally, the speech waveform is generated by feeding the predicted representations into a neural vocoder. Such a paradigm allows us to go beyond spectral and parametric changes of the signal, and model non-verbal vocalizations, such as laughter insertion, yawning removal, etc. We demonstrate objectively and subjectively that the proposed method is vastly superior to current approaches and even beats text-based systems in ter
Authors
(none)
Tags
Stats
Related papers
- Nonparallel Emotional Speech Conversion (2018)11.08
- In-the-wild Speech Emotion Conversion Using Disentangled Self-supervised Representations And Neural Vocoder-based Resynthesis (2023)0.00
- Converting Anyone's Emotion: Towards Speaker-independent Emotional Voice Conversion (2020)11.39
- Multi-speaker Emotion Conversion Via Latent Variable Regularization And A Chained Encoder-decoder-predictor Network (2020)5.84
- EMOCONV-DIFF: Diffusion-based Speech Emotion Conversion For Non-parallel And In-the-wild Data (2023)5.84
- Seen And Unseen Emotional Style Transfer For Voice Conversion With A New Emotional Speech Dataset (2020)16.34
- A Change Of Heart: Improving Speech Emotion Recognition Through Speech-to-text Modality Conversion (2023)0.00
- Learning Multilingual Expressive Speech Representation For Prosody Prediction Without Parallel Data (2023)4.52