Controllable Sequence-to-sequence Neural TTS With LPCNET Backend For Real-time Speech Synthesis On CPU

Abstract

State-of-the-art sequence-to-sequence acoustic networks, that convert a phonetic sequence to a sequence of spectral features with no explicit prosody prediction, generate speech with close to natural quality, when cascaded with neural vocoders, such as Wavenet. However, the combined system is typically too heavy for real-time speech synthesis on a CPU. In this work we present a sequence-to-sequence acoustic network combined with lightweight LPCNet neural vocoder, designed for real-time speech synthesis on a CPU. In addition, the system allows sentence-level pace and expressivity control at inference time. We demonstrate that the proposed system can synthesize high quality 22 kHz speech in real-time on a general-purpose CPU. In terms of MOS score degradation relative to PCM, the system attained as low as 6.1-6.5% for quality and 6.3- 7.0% for expressiveness, reaching equivalent or better quality when compared to a similar system with a Wavenet vocoder backend.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyshechtman2020controllable

Related papers

• High Quality Streaming Speech Synthesis With Low, Sentence-length-independent Latency (2021)8.60
• High Quality, Lightweight And Adaptable TTS Using Lpcnet (2019)10.97
• A Real-time Wideband Neural Vocoder At 1.6 Kb/s Using Lpcnet (2019)12.61
• Lpcnet: Improving Neural Speech Synthesis Through Linear Prediction (2018)0.00
• Neural Speech Synthesis On A Shoestring: Improving The Efficiency Of Lpcnet (2022)5.84
• End-to-end Lpcnet: A Neural Vocoder With Fully-differentiable LPC Estimation (2022)7.16
• Sequence To Sequence Neural Speech Synthesis With Prosody Modification Capabilities (2019)9.59
• Improving Lpcnet-based Text-to-speech With Linear Prediction-structured Mixture Density Network (2020)5.24