Singing Voice Conversion With Accompaniment Using Self-supervised Representation-based Melody Features

Abstract

Melody preservation is crucial in singing voice conversion (SVC). However, in many scenarios, audio is often accompanied with background music (BGM), which can cause audio distortion and interfere with the extraction of melody and other key features, significantly degrading SVC performance. Previous methods have attempted to address this by using more robust neural network-based melody extractors, but their performance drops sharply in the presence of complex accompaniment. Other approaches involve performing source separation before conversion, but this often introduces noticeable artifacts, leading to a significant drop in conversion quality and increasing the user's operational costs. To address these issues, we introduce a novel SVC method that uses self-supervised representation-based melody features to improve melody modeling accuracy in the presence of BGM. In our experiments, we compare the effectiveness of different self-supervised learning (SSL) models for melody extraction a

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keychen2025singing

Related papers

• Robustsvc: Hubert-based Melody Extractor And Adversarial Learning For Robust Singing Voice Conversion (2024)3.58
• Ppg-based Singing Voice Conversion With Adversarial Representation Learning (2020)9.76
• Leveraging Diverse Semantic-based Audio Pretrained Models For Singing Voice Conversion (2023)0.00
• Neural Concatenative Singing Voice Conversion: Rethinking Concatenation-based Approach For One-shot Singing Voice Conversion (2023)7.50
• Adversarial Speaker Disentanglement Using Unannotated External Data For Self-supervised Representation Based Voice Conversion (2023)6.34
• A Melody-unsupervision Model For Singing Voice Synthesis (2021)5.84
• Toward Expressive Singing Voice Correction: On Perceptual Validity Of Evaluation Metrics For Vocal Melody Extraction (2020)0.00
• Semi-supervised Learning Using Teacher-student Models For Vocal Melody Extraction (2020)0.00