Joint Learning Of Wording And Formatting For Singable Melody-to-lyric Generation

Abstract

Despite progress in melody-to-lyric generation, a substantial singability gap remains between machine-generated lyrics and those written by human lyricists. In this work, we aim to narrow this gap by jointly learning both wording and formatting for melody-to-lyric generation. After general-domain pretraining, our model acquires length awareness through an self-supervised stage trained on a large text-only lyric corpus. During supervised melody-to-lyric training, we introduce multiple auxiliary supervision objective informed by musicological findings on melody--lyric relationships, encouraging the model to capture fine-grained prosodic and structural patterns. Compared with na\"ive fine-tuning, our approach improves adherence to line-count and syllable-count requirements by 3.8% and 21.4% absolute, respectively, without degrading text quality. In human evaluation, it achieves 42.2% and 74.2% relative gains in overall quality over two task-specific baselines, underscoring the importance

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyou2023joint

Related papers

• Unsupervised Melody-to-lyric Generation (2023)0.00
• Songglm: Lyric-to-melody Generation With 2D Alignment Encoding And Multi-task Pre-training (2024)3.58
• Interpretable Melody Generation From Lyrics With Discrete-valued Adversarial Training (2022)6.34
• A Melody-unsupervision Model For Singing Voice Synthesis (2021)5.84
• Songmass: Automatic Song Writing With Pre-training And Alignment Constraint (2020)11.39
• Melody-conditioned Lyrics Generation With Seqgans (2020)7.50
• A Syllable-structured, Contextually-based Conditionally Generation Of Chinese Lyrics (2019)7.16
• Conditional LSTM-GAN For Melody Generation From Lyrics (2019)14.69