Schrodinger Bridges Beat Diffusion Models On Text-to-speech Synthesis

Abstract

In text-to-speech (TTS) synthesis, diffusion models have achieved promising generation quality. However, because of the pre-defined data-to-noise diffusion process, their prior distribution is restricted to a noisy representation, which provides little information of the generation target. In this work, we present a novel TTS system, Bridge-TTS, making the first attempt to substitute the noisy Gaussian prior in established diffusion-based TTS methods with a clean and deterministic one, which provides strong structural information of the target. Specifically, we leverage the latent representation obtained from text input as our prior, and build a fully tractable Schrodinger bridge between it and the ground-truth mel-spectrogram, leading to a data-to-data process. Moreover, the tractability and flexibility of our formulation allow us to empirically study the design spaces such as noise schedules, as well as to develop stochastic and deterministic samplers. Experimental results on the LJ-

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keychen2023schrodinger

Related papers

• Diffusion-based Speech Enhancement With Schr\"odinger Bridge And Symmetric Noise Schedule (2024)0.00
• Reflow-tts: A Rectified Flow Model For High-fidelity Text-to-speech (2023)7.50
• High-fidelity Speech Synthesis With Minimal Supervision: All Using Diffusion Models (2023)5.24
• Robust Speech Recognition With Schr\"odinger Bridge-based Speech Enhancement (2025)2.26
• Naturalspeech 2: Latent Diffusion Models Are Natural And Zero-shot Speech And Singing Synthesizers (2023)0.00
• Diffusion Synthesizer For Efficient Multilingual Speech To Speech Translation (2024)0.00
• Speech Enhancement And Dereverberation With Diffusion-based Generative Models (2022)23.51
• Grad-stylespeech: Any-speaker Adaptive Text-to-speech Synthesis With Diffusion Models (2022)0.00