End-to-end Training Of Time Domain Audio Separation And Recognition

Abstract

The rising interest in single-channel multi-speaker speech separation sparked development of End-to-End (E2E) approaches to multi-speaker speech recognition. However, up until now, state-of-the-art neural network-based time domain source separation has not yet been combined with E2E speech recognition. We here demonstrate how to combine a separation module based on a Convolutional Time domain Audio Separation Network (Conv-TasNet) with an E2E speech recognizer and how to train such a model jointly by distributing it over multiple GPUs or by approximating truncated back-propagation for the convolutional front-end. To put this work into perspective and illustrate the complexity of the design space, we provide a compact overview of single-channel multi-speaker recognition systems. Our experiments show a word error rate of 11.0% on WSJ0-2mix and indicate that our joint time domain model can yield substantial improvements over cascade DNN-HMM and monolithic E2E frequency domain systems prop

Stats

• citations23
• S2 citations—
• github stars0
• HF likes0
• heat score10.35
• arxiv keyvonneumann2019end

Related papers

• Tasnet: Time-domain Audio Separation Network For Real-time, Single-channel Speech Separation (2017)20.16
• Conv-tasnet: Surpassing Ideal Time-frequency Magnitude Masking For Speech Separation (2018)24.08
• End-to-end Multi-channel Speech Separation (2019)0.00
• End-to-end Networks For Supervised Single-channel Speech Separation (2018)0.00
• Inter-channel Conv-tasnet For Multichannel Speech Enhancement (2021)0.00
• Improving Voice Separation By Incorporating End-to-end Speech Recognition (2019)0.00
• Multi-scale Feature Fusion Transformer Network For End-to-end Single Channel Speech Separation (2022)0.00
• EEND-SS: Joint End-to-end Neural Speaker Diarization And Speech Separation For Flexible Number Of Speakers (2022)10.35