Learning To Enhance Or Not: Neural Network-based Switching Of Enhanced And Observed Signals For Overlapping Speech Recognition

Abstract

The combination of a deep neural network (DNN) -based speech enhancement (SE) front-end and an automatic speech recognition (ASR) back-end is a widely used approach to implement overlapping speech recognition. However, the SE front-end generates processing artifacts that can degrade the ASR performance. We previously found that such performance degradation can occur even under fully overlapping conditions, depending on the signal-to-interference ratio (SIR) and signal-to-noise ratio (SNR). To mitigate the degradation, we introduced a rule-based method to switch the ASR input between the enhanced and observed signals, which showed promising results. However, the rule's optimality was unclear because it was heuristically designed and based only on SIR and SNR values. In this work, we propose a DNN-based switching method that directly estimates whether ASR will perform better on the enhanced or observed signals. We also introduce soft-switching that computes a weighted sum of the enhanced

Stats

• citations22
• S2 citations—
• github stars0
• HF likes0
• heat score10.21
• arxiv keysato2022learning

Related papers

• Bridging The Gap: Integrating Pre-trained Speech Enhancement And Recognition Models For Robust Speech Recognition (2024)7.50
• Reinforcement Learning Based Speech Enhancement For Robust Speech Recognition (2018)11.08
• Towards Decoupling Frontend Enhancement And Backend Recognition In Monaural Robust ASR (2024)4.52
• Monaural Speech Enhancement Using Deep Neural Networks By Maximizing A Short-time Objective Intelligibility Measure (2018)11.76
• Joint Training Of Speech Enhancement And Self-supervised Model For Noise-robust ASR (2022)0.00
• Snri Target Training For Joint Speech Enhancement And Recognition (2021)8.82
• Overlapped Speech Recognition From A Jointly Learned Multi-channel Neural Speech Extraction And Representation (2019)0.00
• Improving Noise Robust Automatic Speech Recognition With Single-channel Time-domain Enhancement Network (2020)13.88