From Speaker Verification To Multispeaker Speech Synthesis, Deep Transfer With Feedback Constraint
2020 · Zexin Cai, Chuxiong Zhang, Ming Li
Abstract
High-fidelity speech can be synthesized by end-to-end text-to-speech models in recent years. However, accessing and controlling speech attributes such as speaker identity, prosody, and emotion in a text-to-speech system remains a challenge. This paper presents a system involving feedback constraint for multispeaker speech synthesis. We manage to enhance the knowledge transfer from the speaker verification to the speech synthesis by engaging the speaker verification network. The constraint is taken by an added loss related to the speaker identity, which is centralized to improve the speaker similarity between the synthesized speech and its natural reference audio. The model is trained and evaluated on publicly available datasets. Experimental results, including visualization on speaker embedding space, show significant improvement in terms of speaker identity cloning in the spectrogram level. Synthesized samples are available online for listening. (https://caizexin.github.io/mlspk-syn-s
Authors
(none)
Tags
Stats
Related papers
- Speaker Verification-derived Loss And Data Augmentation For Dnn-based Multispeaker Speech Synthesis (2021)3.58
- Speaker Verification In Multi-speaker Environments Using Temporal Feature Fusion (2022)0.00
- Transfer Learning From Speaker Verification To Multispeaker Text-to-speech Synthesis (2018)0.00
- Voice Cloning: A Multi-speaker Text-to-speech Synthesis Approach Based On Transfer Learning (2021)0.00
- Zero-shot Multi-speaker Text-to-speech With State-of-the-art Neural Speaker Embeddings (2019)15.67
- Leveraging ASR Pretrained Conformers For Speaker Verification Through Transfer Learning And Knowledge Distillation (2023)10.74
- Unispeaker: A Unified Approach For Multimodality-driven Speaker Generation (2025)2.26
- Multi-speaker Expressive Speech Synthesis Via Multiple Factors Decoupling (2022)0.00