Crossspeech: Speaker-independent Acoustic Representation For Cross-lingual Speech Synthesis

Abstract

While recent text-to-speech (TTS) systems have made remarkable strides toward human-level quality, the performance of cross-lingual TTS lags behind that of intra-lingual TTS. This gap is mainly rooted from the speaker-language entanglement problem in cross-lingual TTS. In this paper, we propose CrossSpeech which improves the quality of cross-lingual speech by effectively disentangling speaker and language information in the level of acoustic feature space. Specifically, CrossSpeech decomposes the speech generation pipeline into the speaker-independent generator (SIG) and speaker-dependent generator (SDG). The SIG produces the speaker-independent acoustic representation which is not biased to specific speaker distributions. On the other hand, the SDG models speaker-dependent speech variation that characterizes speaker attributes. By handling each information separately, CrossSpeech can obtain disentangled speaker and language representations. From the experiments, we verify that CrossSp

Stats

• citations8
• S2 citations—
• github stars0
• HF likes0
• heat score7.16
• arxiv keykim2023crossspeech

Related papers

• Crossspeech++: Cross-lingual Speech Synthesis With Decoupled Language And Speaker Generation (2024)0.00
• Learning To Speak Fluently In A Foreign Language: Multilingual Speech Synthesis And Cross-language Voice Cloning (2019)15.03
• Cross-lingual Multi-speaker Text-to-speech Synthesis For Voice Cloning Without Using Parallel Corpus For Unseen Speakers (2019)0.00
• Crossvoice: Crosslingual Prosody Preserving Cascade-s2st Using Transfer Learning (2024)0.00
• Synth2aug: Cross-domain Speaker Recognition With TTS Synthesized Speech (2020)6.77
• Exploring Timbre Disentanglement In Non-autoregressive Cross-lingual Text-to-speech (2021)2.26
• Building Multi Lingual TTS Using Cross Lingual Voice Conversion (2020)0.00
• Cross-lingual Low Resource Speaker Adaptation Using Phonological Features (2021)5.24