Conditional LSTM-GAN For Melody Generation From Lyrics

Abstract

Melody generation from lyrics has been a challenging research issue in the field of artificial intelligence and music, which enables to learn and discover latent relationship between interesting lyrics and accompanying melody. Unfortunately, the limited availability of paired lyrics-melody dataset with alignment information has hindered the research progress. To address this problem, we create a large dataset consisting of 12,197 MIDI songs each with paired lyrics and melody alignment through leveraging different music sources where alignment relationship between syllables and music attributes is extracted. Most importantly, we propose a novel deep generative model, conditional Long Short-Term Memory - Generative Adversarial Network (LSTM-GAN) for melody generation from lyrics, which contains a deep LSTM generator and a deep LSTM discriminator both conditioned on lyrics. In particular, lyrics-conditioned melody and alignment relationship between syllables of given lyrics and notes of p

Stats

• citations90
• S2 citations—
• github stars0
• HF likes0
• heat score14.69
• arxiv keyyu2019conditional

Related papers

• Interpretable Melody Generation From Lyrics With Discrete-valued Adversarial Training (2022)6.34
• Melody-conditioned Lyrics Generation With Seqgans (2020)7.50
• Unsupervised Melody-to-lyric Generation (2023)0.00
• Songglm: Lyric-to-melody Generation With 2D Alignment Encoding And Multi-task Pre-training (2024)3.58
• Neural Melody Composition From Lyrics (2018)9.59
• CSL-L2M: Controllable Song-level Lyric-to-melody Generation Based On Conditional Transformer With Fine-grained Lyric And Musical Controls (2024)2.26
• A Syllable-structured, Contextually-based Conditionally Generation Of Chinese Lyrics (2019)7.16
• Joint Learning Of Wording And Formatting For Singable Melody-to-lyric Generation (2023)0.00