S2VC: A Framework For Any-to-any Voice Conversion With Self-supervised Pretrained Representations

Abstract

Any-to-any voice conversion (VC) aims to convert the timbre of utterances from and to any speakers seen or unseen during training. Various any-to-any VC approaches have been proposed like AUTOVC, AdaINVC, and FragmentVC. AUTOVC, and AdaINVC utilize source and target encoders to disentangle the content and speaker information of the features. FragmentVC utilizes two encoders to encode source and target information and adopts cross attention to align the source and target features with similar phonetic content. Moreover, pre-trained features are adopted. AUTOVC used dvector to extract speaker information, and self-supervised learning (SSL) features like wav2vec 2.0 is used in FragmentVC to extract the phonetic content information. Different from previous works, we proposed S2VC that utilizes Self-Supervised features as both source and target features for VC model. Supervised phoneme posteriororgram (PPG), which is believed to be speaker-independent and widely used in VC to extract conten

Stats

• citations42
• S2 citations—
• github stars0
• HF likes0
• heat score12.25
• arxiv keylin2021s2vc

Related papers

• Any-to-one Sequence-to-sequence Voice Conversion Using Self-supervised Discrete Speech Representations (2020)0.00
• A Comparative Study Of Self-supervised Speech Representation Based Voice Conversion (2022)9.76
• S3PRL-VC: Open-source Voice Conversion Framework With Self-supervised Speech Representations (2021)10.97
• DRVC: A Framework Of Any-to-any Voice Conversion With Self-supervised Learning (2022)9.59
• Mediumvc: Any-to-any Voice Conversion Using Synthetic Specific-speaker Speeches As Intermedium Features (2021)0.00
• Fragmentvc: Any-to-any Voice Conversion By End-to-end Extracting And Fusing Fine-grained Voice Fragments With Attention (2020)12.54
• Selfvc: Voice Conversion With Iterative Refinement Using Self Transformations (2023)0.00
• ACE-VC: Adaptive And Controllable Voice Conversion Using Explicitly Disentangled Self-supervised Speech Representations (2023)0.00