Embedding A Differentiable Mel-cepstral Synthesis Filter To A Neural Speech Synthesis System

Abstract

This paper integrates a classic mel-cepstral synthesis filter into a modern neural speech synthesis system towards end-to-end controllable speech synthesis. Since the mel-cepstral synthesis filter is explicitly embedded in neural waveform models in the proposed system, both voice characteristics and the pitch of synthesized speech are highly controlled via a frequency warping parameter and fundamental frequency, respectively. We implement the mel-cepstral synthesis filter as a differentiable and GPU-friendly module to enable the acoustic and waveform models in the proposed system to be simultaneously optimized in an end-to-end manner. Experiments show that the proposed system improves speech quality from a baseline system maintaining controllability. The core PyTorch modules used in the experiments will be publicly available on GitHub.

Stats

• citations4
• S2 citations—
• github stars0
• HF likes0
• heat score5.24
• arxiv keyyoshimura2022embedding

Related papers

• Differentiable WORLD Synthesizer-based Neural Vocoder With Application To End-to-end Audio Style Transfer (2022)0.00
• Speech Synthesis And Control Using Differentiable DSP (2020)0.00
• Speech Waveform Synthesis From MFCC Sequences With Generative Adversarial Networks (2018)12.25
• GELP: Gan-excited Linear Prediction For Speech Synthesis From Mel-spectrogram (2019)10.74
• Hifi-glot: High-fidelity Neural Formant Synthesis With Differentiable Resonant Filters (2024)0.00
• On Using Backpropagation For Speech Texture Generation And Voice Conversion (2017)7.16
• Ultra-lightweight Neural Differential DSP Vocoder For High Quality Speech Synthesis (2024)5.24
• Neural Music Synthesis For Flexible Timbre Control (2018)9.92