Improving Voice Separation By Incorporating End-to-end Speech Recognition

Abstract

Despite recent advances in voice separation methods, many challenges remain in realistic scenarios such as noisy recording and the limits of available data. In this work, we propose to explicitly incorporate the phonetic and linguistic nature of speech by taking a transfer learning approach using an end-to-end automatic speech recognition (E2EASR) system. The voice separation is conditioned on deep features extracted from E2EASR to cover the long-term dependence of phonetic aspects. Experimental results on speech separation and enhancement task on the AVSpeech dataset show that the proposed method significantly improves the signal-to-distortion ratio over the baseline model and even outperforms an audio visual model, that utilizes visual information of lip movements.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keytakahashi2019improving

Related papers

• Investigation Of Practical Aspects Of Single Channel Speech Separation For ASR (2021)7.81
• End-to-end Training Of Time Domain Audio Separation And Recognition (2019)10.35
• An Overview Of Deep-learning-based Audio-visual Speech Enhancement And Separation (2020)18.31
• End-to-end Dereverberation, Beamforming, And Speech Recognition With Improved Numerical Stability And Advanced Frontend (2021)10.97
• End-to-end Multi-channel Speech Separation (2019)0.00
• Unified End-to-end Speech Recognition And Endpointing For Fast And Efficient Speech Systems (2022)5.24
• Espnet-se: End-to-end Speech Enhancement And Separation Toolkit Designed For Asr Integration (2020)13.55
• Audio-visual Multi-channel Speech Separation, Dereverberation And Recognition (2022)6.77