Beyond Voice Identity Conversion: Manipulating Voice Attributes By Adversarial Learning Of Structured Disentangled Representations

Abstract

Voice conversion (VC) consists of digitally altering the voice of an individual to manipulate part of its content, primarily its identity, while maintaining the rest unchanged. Research in neural VC has accomplished considerable breakthroughs with the capacity to falsify a voice identity using a small amount of data with a highly realistic rendering. This paper goes beyond voice identity and presents a neural architecture that allows the manipulation of voice attributes (e.g., gender and age). Leveraging the latest advances on adversarial learning of structured speech representation, a novel structured neural network is proposed in which multiple auto-encoders are used to encode speech as a set of idealistically independent linguistic and extra-linguistic representations, which are learned adversariarly and can be manipulated during VC. Moreover, the proposed architecture is time-synchronized so that the original voice timing is preserved during conversion which allows lip-sync applica

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keybenaroya2021beyond

Related papers

• Voice Reenactment With F0 And Timing Constraints And Adversarial Learning Of Conversions (2021)2.26
• Generative Adversarial Network Based Voice Conversion: Techniques, Challenges, And Recent Advancements (2025)0.00
• ACE-VC: Adaptive And Controllable Voice Conversion Using Explicitly Disentangled Self-supervised Speech Representations (2023)0.00
• Assem-vc: Realistic Voice Conversion By Assembling Modern Speech Synthesis Techniques (2021)11.64
• An Adaptive Learning Based Generative Adversarial Network For One-to-one Voice Conversion (2021)10.61
• Discrete Unit Based Masking For Improving Disentanglement In Voice Conversion (2024)0.00
• An Overview Of Voice Conversion And Its Challenges: From Statistical Modeling To Deep Learning (2020)18.53
• Zero-shot Voice Conversion Via Self-supervised Prosody Representation Learning (2021)6.34