Emotion Rendering For Conversational Speech Synthesis With Heterogeneous Graph-based Context Modeling

Abstract

Conversational Speech Synthesis (CSS) aims to accurately express an utterance with the appropriate prosody and emotional inflection within a conversational setting. While recognising the significance of CSS task, the prior studies have not thoroughly investigated the emotional expressiveness problems due to the scarcity of emotional conversational datasets and the difficulty of stateful emotion modeling. In this paper, we propose a novel emotional CSS model, termed ECSS, that includes two main components: 1) to enhance emotion understanding, we introduce a heterogeneous graph-based emotional context modeling mechanism, which takes the multi-source dialogue history as input to model the dialogue context and learn the emotion cues from the context; 2) to achieve emotion rendering, we employ a contrastive learning-based emotion renderer module to infer the accurate emotion style for the target utterance. To address the issue of data scarcity, we meticulously create emotional labels in ter

Stats

• citations18
• S2 citations—
• github stars59
• HF likes0
• heat score13.15
• arxiv keyliu2023emotion

Related papers

• CONCSS: Contrastive-based Context Comprehension For Dialogue-appropriate Prosody In Conversational Speech Synthesis (2023)0.00
• Diffcss: Diverse And Expressive Conversational Speech Synthesis With Diffusion Models (2025)0.00
• Intra- And Inter-modal Context Interaction Modeling For Conversational Speech Synthesis (2024)4.53
• Emphasis Rendering For Conversational Text-to-speech With Multi-modal Multi-scale Context Modeling (2024)0.00
• Msemotts: Multi-scale Emotion Transfer, Prediction, And Control For Emotional Speech Synthesis (2022)13.97
• Self-supervised Context-aware Style Representation For Expressive Speech Synthesis (2022)6.34
• Leveraging Real Conversational Data For Multi-channel Continuous Speech Separation (2022)0.00
• Emosphere-tts: Emotional Style And Intensity Modeling Via Spherical Emotion Vector For Controllable Emotional Text-to-speech (2024)10.35