Vevo: Controllable Zero-shot Voice Imitation With Self-supervised Disentanglement
2025 · Xueyao Zhang, Xiaohui Zhang, Kainan Peng, et al.
Abstract
The imitation of voice, targeted on specific speech attributes such as timbre and speaking style, is crucial in speech generation. However, existing methods rely heavily on annotated data, and struggle with effectively disentangling timbre and style, leading to challenges in achieving controllable generation, especially in zero-shot scenarios. To address these issues, we propose Vevo, a versatile zero-shot voice imitation framework with controllable timbre and style. Vevo operates in two core stages: (1) Content-Style Modeling: Given either text or speech's content tokens as input, we utilize an autoregressive transformer to generate the content-style tokens, which is prompted by a style reference; (2) Acoustic Modeling: Given the content-style tokens as input, we employ a flow-matching transformer to produce acoustic representations, which is prompted by a timbre reference. To obtain the content and content-style tokens of speech, we design a fully self-supervised approach that progre
Authors
(none)
Tags
Stats
Related papers
- Vevo2: A Unified And Controllable Framework For Speech And Singing Voice Generation (2025)0.00
- Improving Zero-shot Voice Style Transfer Via Disentangled Representation Learning (2021)0.00
- Seeing Your Speech Style: A Novel Zero-shot Identity-disentanglement Face-based Voice Conversion (2024)4.52
- ZSVC: Zero-shot Style Voice Conversion With Disentangled Latent Diffusion Models And Adversarial Training (2025)0.00
- Robust Disentangled Variational Speech Representation Learning For Zero-shot Voice Conversion (2022)10.97
- ACE-VC: Adaptive And Controllable Voice Conversion Using Explicitly Disentangled Self-supervised Speech Representations (2023)0.00
- Stablevc: Style Controllable Zero-shot Voice Conversion With Conditional Flow Matching (2024)7.81
- Zero-shot Voice Conversion Via Self-supervised Prosody Representation Learning (2021)6.34