Generative Modeling For Low Dimensional Speech Attributes With Neural Spline Flows

Abstract

Despite recent advances in generative modeling for text-to-speech synthesis, these models do not yet have the same fine-grained adjustability of pitch-conditioned deterministic models such as FastPitch and FastSpeech2. Pitch information is not only low-dimensional, but also discontinuous, making it particularly difficult to model in a generative setting. Our work explores several techniques for handling the aforementioned issues in the context of Normalizing Flow models. We also find this problem to be very well suited for Neural Spline flows, which is a highly expressive alternative to the more common affine-coupling mechanism in Normalizing Flows.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyshih2022generative

Related papers

• Generative Pre-training For Speech With Flow Matching (2023)0.00
• Latent Normalizing Flows For Discrete Sequences (2019)0.00
• Improving Multi-speaker TTS Prosody Variance With A Residual Encoder And Normalizing Flows (2021)0.00
• Flowvocoder: A Small Footprint Neural Vocoder Based Normalizing Flow For Speech Synthesis (2021)0.00
• Stochastic Pitch Prediction Improves The Diversity And Naturalness Of Speech In Glow-tts (2023)3.58
• Predicting Phoneme-level Prosody Latents Using AR And Flow-based Prior Networks For Expressive Speech Synthesis (2022)0.00
• Comparing Normalizing Flows And Diffusion Models For Prosody And Acoustic Modelling In Text-to-speech (2023)0.00
• Improved Normalizing Flow-based Speech Enhancement Using An All-pole Gammatone Filterbank For Conditional Input Representation (2022)0.00