Sequence-to-sequence Singing Voice Synthesis With Perceptual Entropy Loss

Abstract

The neural network (NN) based singing voice synthesis (SVS) systems require sufficient data to train well and are prone to over-fitting due to data scarcity. However, we often encounter data limitation problem in building SVS systems because of high data acquisition and annotation costs. In this work, we propose a Perceptual Entropy (PE) loss derived from a psycho-acoustic hearing model to regularize the network. With a one-hour open-source singing voice database, we explore the impact of the PE loss on various mainstream sequence-to-sequence models, including the RNN-based, transformer-based, and conformer-based models. Our experiments show that the PE loss can mitigate the over-fitting problem and significantly improve the synthesized singing quality reflected in objective and subjective evaluations.

Stats

• citations18
• S2 citations—
• github stars0
• HF likes0
• heat score9.59
• arxiv keyshi2020sequence

Related papers

• Singaug: Data Augmentation For Singing Voice Synthesis With Cycle-consistent Training Strategy (2022)7.16
• Visinger2+: End-to-end Singing Voice Synthesis Augmented By Self-supervised Learning Representation (2024)4.52
• A Comparative Analysis Of Latent Regressor Losses For Singing Voice Conversion (2023)0.00
• Self-supervised Singing Voice Pre-training Towards Speech-to-singing Conversion (2024)0.00
• An Empirical Study On End-to-end Singing Voice Synthesis With Encoder-decoder Architectures (2021)0.00
• Adversarially Trained Multi-singer Sequence-to-sequence Singing Synthesizer (2020)7.81
• Towards Improving The Expressiveness Of Singing Voice Synthesis With BERT Derived Semantic Information (2023)0.00
• Singing Voice Data Scaling-up: An Introduction To Ace-opencpop And Ace-kising (2024)15.48