A Benchmark Of Dynamical Variational Autoencoders Applied To Speech Spectrogram Modeling

Abstract

The Variational Autoencoder (VAE) is a powerful deep generative model that is now extensively used to represent high-dimensional complex data via a low-dimensional latent space learned in an unsupervised manner. In the original VAE model, input data vectors are processed independently. In recent years, a series of papers have presented different extensions of the VAE to process sequential data, that not only model the latent space, but also model the temporal dependencies within a sequence of data vectors and corresponding latent vectors, relying on recurrent neural networks. We recently performed a comprehensive review of those models and unified them into a general class called Dynamical Variational Autoencoders (DVAEs). In the present paper, we present the results of an experimental benchmark comparing six of those DVAE models on the speech analysis-resynthesis task, as an illustration of the high potential of DVAEs for speech modeling.

Stats

• citations7
• S2 citations—
• github stars0
• HF likes0
• heat score6.77
• arxiv keybie2021a

Related papers

• Unsupervised Speech Enhancement Using Dynamical Variational Auto-encoders (2021)13.28
• A Statistically Principled And Computationally Efficient Approach To Speech Enhancement Using Variational Autoencoders (2019)9.23
• A Multimodal Dynamical Variational Autoencoder For Audiovisual Speech Representation Learning (2023)2.26
• Convolutional Variational Autoencoders For Spectrogram Compression In Automatic Speech Recognition (2024)0.00
• RAVE: A Variational Autoencoder For Fast And High-quality Neural Audio Synthesis (2021)0.00
• Learning And Controlling The Source-filter Representation Of Speech With A Variational Autoencoder (2022)7.50
• Variational Autoencoders For Learning Latent Representations Of Speech Emotion: A Preliminary Study (2017)13.11
• Audio-visual Speech Enhancement Using Conditional Variational Auto-encoders (2019)13.65