Unsupervised Speech Enhancement Using Dynamical Variational Auto-encoders

Abstract

Dynamical variational autoencoders (DVAEs) are a class of deep generative models with latent variables, dedicated to model time series of high-dimensional data. DVAEs can be considered as extensions of the variational autoencoder (VAE) that include temporal dependencies between successive observed and/or latent vectors. Previous work has shown the interest of using DVAEs over the VAE for speech spectrograms modeling. Independently, the VAE has been successfully applied to speech enhancement in noise, in an unsupervised noise-agnostic set-up that requires neither noise samples nor noisy speech samples at training time, but only requires clean speech signals. In this paper, we extend these works to DVAE-based single-channel unsupervised speech enhancement, hence exploiting both speech signals unsupervised representation learning and dynamics modeling. We propose an unsupervised speech enhancement algorithm that combines a DVAE speech prior pre-trained on clean speech signals with a noise

Stats

• citations58
• S2 citations—
• github stars0
• HF likes0
• heat score13.28
• arxiv keybie2021unsupervised

Related papers

• Unsupervised Speech Enhancement With Deep Dynamical Generative Speech And Noise Models (2023)0.00
• A Benchmark Of Dynamical Variational Autoencoders Applied To Speech Spectrogram Modeling (2021)6.77
• A Statistically Principled And Computationally Efficient Approach To Speech Enhancement Using Variational Autoencoders (2019)9.23
• Statistical Speech Enhancement Based On Probabilistic Integration Of Variational Autoencoder And Non-negative Matrix Factorization (2017)15.00
• Audio-visual Speech Enhancement Using Conditional Variational Auto-encoders (2019)13.65
• A Multimodal Dynamical Variational Autoencoder For Audiovisual Speech Representation Learning (2023)2.26
• A Recurrent Variational Autoencoder For Speech Enhancement (2019)13.97
• I-DCCRN-VAE: An Improved Deep Representation Learning Framework For Complex Vae-based Single-channel Speech Enhancement (2025)0.00