Learning Robust Speech Representation With An Articulatory-regularized Variational Autoencoder

Abstract

It is increasingly considered that human speech perception and production both rely on articulatory representations. In this paper, we investigate whether this type of representation could improve the performances of a deep generative model (here a variational autoencoder) trained to encode and decode acoustic speech features. First we develop an articulatory model able to associate articulatory parameters describing the jaw, tongue, lips and velum configurations with vocal tract shapes and spectral features. Then we incorporate these articulatory parameters into a variational autoencoder applied on spectral features by using a regularization technique that constraints part of the latent space to follow articulatory trajectories. We show that this articulatory constraint improves model training by decreasing time to convergence and reconstruction loss at convergence, and yields better performance in a speech denoising task.

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keygeorges2021learning

Related papers

• Learning And Controlling The Source-filter Representation Of Speech With A Variational Autoencoder (2022)7.50
• Articulatory Representation Learning Via Joint Factor Analysis And Neural Matrix Factorization (2022)7.50
• Variational Auto-encoder Based Variability Encoding For Dysarthric Speech Recognition (2022)7.16
• Deep Encoder-decoder Models For Unsupervised Learning Of Controllable Speech Synthesis (2018)0.00
• A Statistically Principled And Computationally Efficient Approach To Speech Enhancement Using Variational Autoencoders (2019)9.23
• RAVE: A Variational Autoencoder For Fast And High-quality Neural Audio Synthesis (2021)0.00
• Expressive Speech Synthesis Via Modeling Expressions With Variational Autoencoder (2018)13.88
• Learning Latent Representations For Speech Generation And Transformation (2017)13.50