Unsupervised Low Latency Speech Enhancement With RT-GCC-NMF

Abstract

In this paper, we present RT-GCC-NMF: a real-time (RT), two-channel blind speech enhancement algorithm that combines the non-negative matrix factorization (NMF) dictionary learning algorithm with the generalized cross-correlation (GCC) spatial localization method. Using a pre-learned universal NMF dictionary, RT-GCC-NMF operates in a frame-by-frame fashion by associating individual dictionary atoms to target speech or background interference based on their estimated time-delay of arrivals (TDOA). We evaluate RT-GCC-NMF on two-channel mixtures of speech and real-world noise from the Signal Separation and Evaluation Campaign (SiSEC). We demonstrate that this approach generalizes to new speakers, acoustic environments, and recording setups from very little training data, and outperforms all but one of the algorithms from the SiSEC challenge in terms of overall Perceptual Evaluation methods for Audio Source Separation (PEASS) scores and compares favourably to the ideal binary mask baseline

Stats

• citations18
• S2 citations—
• github stars0
• HF likes0
• heat score9.59
• arxiv keywood2019unsupervised

Related papers

• Supervised And Unsupervised Speech Enhancement Using Nonnegative Matrix Factorization (2017)18.80
• Unsupervised Speech Enhancement Based On Multichannel Nmf-informed Beamforming For Noise-robust Automatic Speech Recognition (2019)13.23
• Superm2m: Supervised And Mixture-to-mixture Co-learning For Speech Enhancement And Noise-robust ASR (2024)5.24
• Lightweight Speech Enhancement Guided Target Speech Extraction In Noisy Multi-speaker Scenarios (2025)0.00
• Thlnet: Two-stage Heterogeneous Lightweight Network For Monaural Speech Enhancement (2023)0.00
• A Speech Enhancement Algorithm Based On Non-negative Hidden Markov Model And Kullback-leibler Divergence (2020)5.84
• Dnn-free Low-latency Adaptive Speech Enhancement Based On Frame-online Beamforming Powered By Block-online Fastmnmf (2022)0.00
• Unifying Speech Enhancement And Separation With Gradient Modulation For End-to-end Noise-robust Speech Separation (2023)0.00