Modeling The Rhythm From Lyrics For Melody Generation Of Pop Song

Abstract

Creating a pop song melody according to pre-written lyrics is a typical practice for composers. A computational model of how lyrics are set as melodies is important for automatic composition systems, but an end-to-end lyric-to-melody model would require enormous amounts of paired training data. To mitigate the data constraints, we adopt a two-stage approach, dividing the task into lyric-to-rhythm and rhythm-to-melody modules. However, the lyric-to-rhythm task is still challenging due to its multimodality. In this paper, we propose a novel lyric-to-rhythm framework that includes part-of-speech tags to achieve better text setting, and a Transformer architecture designed to model long-term syllable-to-note associations. For the rhythm-to-melody task, we adapt a proven chord-conditioned melody Transformer, which has achieved state-of-the-art results. Experiments for Chinese lyric-to-melody generation show that the proposed framework is able to model key characteristics of rhythm and pitch

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhang2023modeling

Related papers

• Neural Melody Composition From Lyrics (2018)9.59
• Unsupervised Melody-to-lyric Generation (2023)0.00
• A Syllable-structured, Contextually-based Conditionally Generation Of Chinese Lyrics (2019)7.16
• Joint Learning Of Wording And Formatting For Singable Melody-to-lyric Generation (2023)0.00
• Songglm: Lyric-to-melody Generation With 2D Alignment Encoding And Multi-task Pre-training (2024)3.58
• Conditional LSTM-GAN For Melody Generation From Lyrics (2019)14.69
• CSL-L2M: Controllable Song-level Lyric-to-melody Generation Based On Conditional Transformer With Fine-grained Lyric And Musical Controls (2024)2.26
• Diverse Melody Generation From Chinese Lyrics Via Mutual Information Maximization (2020)0.00