Unsupervised Melody-to-lyric Generation

Abstract

Automatic melody-to-lyric generation is a task in which song lyrics are generated to go with a given melody. It is of significant practical interest and more challenging than unconstrained lyric generation as the music imposes additional constraints onto the lyrics. The training data is limited as most songs are copyrighted, resulting in models that underfit the complicated cross-modal relationship between melody and lyrics. In this work, we propose a method for generating high-quality lyrics without training on any aligned melody-lyric data. Specifically, we design a hierarchical lyric generation framework that first generates a song outline and second the complete lyrics. The framework enables disentanglement of training (based purely on text) from inference (melody-guided text generation) to circumvent the shortage of parallel data. We leverage the segmentation and rhythm alignment between melody and lyrics to compile the given melody into decoding constraints as guidance during i

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keytian2023unsupervised

Related papers

• Joint Learning Of Wording And Formatting For Singable Melody-to-lyric Generation (2023)0.00
• Neural Melody Composition From Lyrics (2018)9.59
• Conditional LSTM-GAN For Melody Generation From Lyrics (2019)14.69
• Songglm: Lyric-to-melody Generation With 2D Alignment Encoding And Multi-task Pre-training (2024)3.58
• Modeling The Rhythm From Lyrics For Melody Generation Of Pop Song (2023)0.00
• Interpretable Melody Generation From Lyrics With Discrete-valued Adversarial Training (2022)6.34
• Melody-conditioned Lyrics Generation With Seqgans (2020)7.50
• A Syllable-structured, Contextually-based Conditionally Generation Of Chinese Lyrics (2019)7.16