Speech And Noise Dual-stream Spectrogram Refine Network With Speech Distortion Loss For Robust Speech Recognition

Abstract

In recent years, the joint training of speech enhancement front-end and automatic speech recognition (ASR) back-end has been widely used to improve the robustness of ASR systems. Traditional joint training methods only use enhanced speech as input for the backend. However, it is difficult for speech enhancement systems to directly separate speech from input due to the diverse types of noise with different intensities. Furthermore, speech distortion and residual noise are often observed in enhanced speech, and the distortion of speech and noise is different. Most existing methods focus on fusing enhanced and noisy features to address this issue. In this paper, we propose a dual-stream spectrogram refine network to simultaneously refine the speech and noise and decouple the noise from the noisy input. Our proposed method can achieve better performance with a relative 8.6% CER reduction.

Stats

• citations4
• S2 citations—
• github stars0
• HF likes0
• heat score5.24
• arxiv keylu2023speech

Related papers

• Gated Recurrent Fusion With Joint Training Framework For Robust End-to-end Speech Recognition (2020)14.55
• Trnet: Two-level Refinement Network Leveraging Speech Enhancement For Noise Robust Speech Emotion Recognition (2024)6.77
• Bridging The Gap: Integrating Pre-trained Speech Enhancement And Recognition Models For Robust Speech Recognition (2024)7.50
• Audio-visual Speech Separation And Dereverberation With A Two-stage Multimodal Network (2019)12.47
• How Does End-to-end Speech Recognition Training Impact Speech Enhancement Artifacts? (2023)7.50
• Effect Of Noise Suppression Losses On Speech Distortion And ASR Performance (2021)10.74
• Towards Decoupling Frontend Enhancement And Backend Recognition In Monaural Robust ASR (2024)4.52
• A Neural Speech Codec For Noise Robust Speech Coding (2023)0.00