Hierarchical Generative Modeling Of Melodic Vocal Contours In Hindustani Classical Music

Abstract

Hindustani music is a performance-driven oral tradition that exhibits the rendition of rich melodic patterns. In this paper, we focus on generative modeling of singers' vocal melodies extracted from audio recordings, as the voice is musically prominent within the tradition. Prior generative work in Hindustani music models melodies as coarse discrete symbols which fails to capture the rich expressive melodic intricacies of singing. Thus, we propose to use a finely quantized pitch contour, as an intermediate representation for hierarchical audio modeling. We propose GaMaDHaNi, a modular two-level hierarchy, consisting of a generative model on pitch contours, and a pitch contour to audio synthesis model. We compare our approach to non-hierarchical audio models and hierarchical models that use a self-supervised intermediate representation, through a listening test and qualitative analysis. We also evaluate audio model's ability to faithfully represent the pitch contour input using Pearson

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyshikarpur2024hierarchical

Related papers

• Audio Segmentation Based On Melodic Style With Hand-crafted Features And With Convolutional Neural Networks (2018)0.00
• Adversarial Multi-task Learning For Disentangling Timbre And Pitch In Singing Voice Synthesis (2022)4.52
• A Melody-unsupervision Model For Singing Voice Synthesis (2021)5.84
• Conditional Variational Autoencoder To Improve Neural Audio Synthesis For Polyphonic Music Sound (2022)0.00
• Emotion-conditioned Melody Harmonization With Hierarchical Variational Autoencoder (2023)5.24
• Unconditional Audio Generation With Generative Adversarial Networks And Cycle Regularization (2020)9.41
• Towards Improving Harmonic Sensitivity And Prediction Stability For Singing Melody Extraction (2023)0.00
• Improving Adversarial Waveform Generation Based Singing Voice Conversion With Harmonic Signals (2022)7.50