Learning Speaker Embedding From Text-to-speech

Abstract

Zero-shot multi-speaker Text-to-Speech (TTS) generates target speaker voices given an input text and the corresponding speaker embedding. In this work, we investigate the effectiveness of the TTS reconstruction objective to improve representation learning for speaker verification. We jointly trained end-to-end Tacotron 2 TTS and speaker embedding networks in a self-supervised fashion. We hypothesize that the embeddings will contain minimal phonetic information since the TTS decoder will obtain that information from the textual input. TTS reconstruction can also be combined with speaker classification to enhance these embeddings further. Once trained, the speaker encoder computes representations for the speaker verification task, while the rest of the TTS blocks are discarded. We investigated training TTS from either manual or ASR-generated transcripts. The latter allows us to train embeddings on datasets without manual transcripts. We compared ASR transcripts and Kaldi phone alignments

Stats

• citations5
• S2 citations—
• github stars0
• HF likes0
• heat score5.84
• arxiv keycho2020learning

Related papers

• Transfer Learning From Speaker Verification To Multispeaker Text-to-speech Synthesis (2018)0.00
• Zero-shot Multi-speaker Text-to-speech With State-of-the-art Neural Speaker Embeddings (2019)15.67
• Semi-supervised Learning For Multi-speaker Text-to-speech Synthesis Using Discrete Speech Representation (2020)5.24
• Voice Imitating Text-to-speech Neural Networks (2018)0.00
• An Analysis On The Effects Of Speaker Embedding Choice In Non Auto-regressive TTS (2023)0.00
• SATTS: Speaker Attractor Text To Speech, Learning To Speak By Learning To Separate (2022)0.00
• Investigation Of Speaker Representation For Target-speaker Speech Processing (2024)4.52
• Enhancing Zero-shot Multi-speaker TTS With Negated Speaker Representations (2024)3.58