Improving Reverberant Speech Separation With Multi-stage Training And Curriculum Learning

Abstract

We present a novel approach that improves the performance of reverberant speech separation. Our approach is based on an accurate geometric acoustic simulator (GAS) which generates realistic room impulse responses (RIRs) by modeling both specular and diffuse reflections. We also propose three training methods - pre-training, multi-stage training and curriculum learning that significantly improve separation quality in the presence of reverberation. We also demonstrate that mixing the synthetic RIRs with a small number of real RIRs during training enhances separation performance. We evaluate our approach on reverberant mixtures generated from real, recorded data (in several different room configurations) from the VOiCES dataset. Our novel approach (curriculum learning+pre-training+multi-stage training) results in a significant relative improvement over prior techniques based on image source method (ISM).

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyaralikatti2021improving

Related papers

• Synthetic Wave-geometric Impulse Responses For Improved Speech Dereverberation (2022)0.00
• A Multi-stage Triple-path Method For Speech Separation In Noisy And Reverberant Environments (2023)2.26
• Convolutive Transfer Function Invariant SDR Training Criteria For Multi-channel Reverberant Speech Separation (2020)0.00
• Audio-visual Speech Separation And Dereverberation With A Two-stage Multimodal Network (2019)12.47
• AV-RIR: Audio-visual Room Impulse Response Estimation (2023)0.00
• RIR-SF: Room Impulse Response Based Spatial Feature For Target Speech Recognition In Multi-channel Multi-speaker Scenarios (2023)0.00
• Towards Improved Room Impulse Response Estimation For Speech Recognition (2022)10.61
• TS-RIR: Translated Synthetic Room Impulse Responses For Speech Augmentation (2021)8.35