Emotion-aligned Generation In Diffusion Text To Speech Models Via Preference-guided Optimization

Abstract

Emotional text-to-speech seeks to convey affect while preserving intelligibility and prosody, yet existing methods rely on coarse labels or proxy classifiers and receive only utterance-level feedback. We introduce Emotion-Aware Stepwise Preference Optimization (EASPO), a post-training framework that aligns diffusion TTS with fine-grained emotional preferences at intermediate denoising steps. Central to our approach is EASPM, a time-conditioned model that scores noisy intermediate speech states and enables automatic preference pair construction. EASPO optimizes generation to match these stepwise preferences, enabling controllable emotional shaping. Experiments show superior performance over existing methods in both expressiveness and naturalness.

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keyshi2025emotion

Related papers

• Emo-dpo: Controllable Emotional Speech Synthesis Through Direct Preference Optimization (2024)9.59
• Emodiff: Intensity Controllable Emotional Text-to-speech With Soft-label Guidance (2022)0.00
• Emomix: Emotion Mixing Via Diffusion Models For Emotional Speech Synthesis (2023)0.00
• Enhancing Emotional Text-to-speech Controllability With Natural Language Guidance Through Contrastive Learning And Diffusion Models (2024)5.24
• Emotional Dimension Control In Language Model-based Text-to-speech: Spanning A Broad Spectrum Of Human Emotions (2024)0.00
• Fine-grained Emotional Control Of Text-to-speech: Learning To Rank Inter- And Intra-class Emotion Intensities (2023)6.77
• Zet-speech: Zero-shot Adaptive Emotion-controllable Text-to-speech Synthesis With Diffusion And Style-based Models (2023)8.09
• Emosphere-tts: Emotional Style And Intensity Modeling Via Spherical Emotion Vector For Controllable Emotional Text-to-speech (2024)10.35