Diffusion-based Generative Modeling With Discriminative Guidance For Streamable Speech Enhancement

Abstract

Diffusion-based generative models (DGMs) have recently attracted attention in speech enhancement research (SE) as previous works showed a remarkable generalization capability. However, DGMs are also computationally intensive, as they usually require many iterations in the reverse diffusion process (RDP), making them impractical for streaming SE systems. In this paper, we propose to use discriminative scores from discriminative models in the first steps of the RDP. These discriminative scores require only one forward pass with the discriminative model for multiple RDP steps, thus greatly reducing computations. This approach also allows for performance improvements. We show that we can trade off between generative and discriminative capabilities as the number of steps with the discriminative score increases. Furthermore, we propose a novel streamable time-domain generative model with an algorithmic latency of 50 ms, which has no significant performance degradation compared to offline mod

Stats

• citations8
• S2 citations—
• github stars0
• HF likes0
• heat score7.16
• arxiv keyli2024diffusion

Related papers

• Speech Enhancement And Dereverberation With Diffusion-based Generative Models (2022)23.51
• Single And Few-step Diffusion For Generative Speech Enhancement (2023)10.21
• Extract And Diffuse: Latent Integration For Improved Diffusion-based Speech And Vocal Enhancement (2024)0.00
• Diffusion-based Speech Enhancement With A Weighted Generative-supervised Learning Loss (2023)0.00
• Analysing Diffusion-based Generative Approaches Versus Discriminative Approaches For Speech Restoration (2022)11.39
• Diffusion-based Speech Enhancement With Joint Generative And Predictive Decoders (2023)9.59
• Gdiffuse: Diffusion-based Speech Enhancement With Noise Model Guidance (2025)0.00
• Real-time Streamable Generative Speech Restoration With Flow Matching (2025)0.00