Inference Skipping For More Efficient Real-time Speech Enhancement With Parallel Rnns

Abstract

Deep neural network (DNN) based speech enhancement models have attracted extensive attention due to their promising performance. However, it is difficult to deploy a powerful DNN in real-time applications because of its high computational cost. Typical compression methods such as pruning and quantization do not make good use of the data characteristics. In this paper, we introduce the Skip-RNN strategy into speech enhancement models with parallel RNNs. The states of the RNNs update intermittently without interrupting the update of the output mask, which leads to significant reduction of computational load without evident audio artifacts. To better leverage the difference between the voice and the noise, we further regularize the skipping strategy with voice activity detection (VAD) guidance, saving more computational load. Experiments on a high-performance speech enhancement model, dual-path convolutional recurrent network (DPCRN), show the superiority of our strategy over strategies l

Stats

• citations23
• S2 citations—
• github stars0
• HF likes0
• heat score10.35
• arxiv keyle2022inference

Related papers

• DPCRN: Dual-path Convolution Recurrent Network For Single Channel Speech Enhancement (2021)14.35
• Dynamic Gated Recurrent Neural Network For Compute-efficient Speech Enhancement (2024)8.35
• PDPCRN: Parallel Dual-path CRN With Bi-directional Inter-branch Interactions For Multi-channel Speech Enhancement (2023)0.00
• Skipconvnet: Skip Convolutional Neural Network For Speech Dereverberation Using Optimally Smoothed Spectral Mapping (2020)10.21
• Dual-path Self-attention RNN For Real-time Speech Enhancement (2020)0.00
• I-DCCRN-VAE: An Improved Deep Representation Learning Framework For Complex Vae-based Single-channel Speech Enhancement (2025)0.00
• Run-time Adaptation Of Neural Beamforming For Robust Speech Dereverberation And Denoising (2024)0.00
• Convolutional Recurrent Neural Network Based Progressive Learning For Monaural Speech Enhancement (2019)0.00