Interpretable Melody Generation From Lyrics With Discrete-valued Adversarial Training

Abstract

Generating melody from lyrics is an interesting yet challenging task in the area of artificial intelligence and music. However, the difficulty of keeping the consistency between input lyrics and generated melody limits the generation quality of previous works. In our proposal, we demonstrate our proposed interpretable lyrics-to-melody generation system which can interact with users to understand the generation process and recreate the desired songs. To improve the reliability of melody generation that matches lyrics, mutual information is exploited to strengthen the consistency between lyrics and generated melodies. Gumbel-Softmax is exploited to solve the non-differentiability problem of generating discrete music attributes by Generative Adversarial Networks (GANs). Moreover, the predicted probabilities output by the generator is utilized to recommend music attributes. Interacting with our lyrics-to-melody generation system, users can listen to the generated AI song as well as recreat

Stats

• citations6
• S2 citations—
• github stars0
• HF likes0
• heat score6.34
• arxiv keyduan2022interpretable

Related papers

• Conditional LSTM-GAN For Melody Generation From Lyrics (2019)14.69
• Melody-conditioned Lyrics Generation With Seqgans (2020)7.50
• Unsupervised Melody-to-lyric Generation (2023)0.00
• Songglm: Lyric-to-melody Generation With 2D Alignment Encoding And Multi-task Pre-training (2024)3.58
• Diverse Melody Generation From Chinese Lyrics Via Mutual Information Maximization (2020)0.00
• Joint Learning Of Wording And Formatting For Singable Melody-to-lyric Generation (2023)0.00
• Neural Melody Composition From Lyrics (2018)9.59
• CSL-L2M: Controllable Song-level Lyric-to-melody Generation Based On Conditional Transformer With Fine-grained Lyric And Musical Controls (2024)2.26