Espnet-se: End-to-end Speech Enhancement And Separation Toolkit Designed For Asr Integration

Abstract

We present ESPnet-SE, which is designed for the quick development of speech enhancement and speech separation systems in a single framework, along with the optional downstream speech recognition module. ESPnet-SE is a new project which integrates rich automatic speech recognition related models, resources and systems to support and validate the proposed front-end implementation (i.e. speech enhancement and separation).It is capable of processing both single-channel and multi-channel data, with various functionalities including dereverberation, denoising and source separation. We provide all-in-one recipes including data pre-processing, feature extraction, training and evaluation pipelines for a wide range of benchmark datasets. This paper describes the design of the toolkit, several important functionalities, especially the speech recognition integration, which differentiates ESPnet-SE from other open source toolkits, and experimental results with major benchmark datasets.

Stats

• citations63
• S2 citations—
• github stars0
• HF likes0
• heat score13.55
• arxiv keyli2020espnet

Related papers

• Espnet-se++: Speech Enhancement For Robust Speech Recognition, Translation, And Understanding (2022)18.72
• Espnet: End-to-end Speech Processing Toolkit (2018)22.17
• The 2020 Espnet Update: New Features, Broadened Applications, Performance Improvements, And Future Plans (2020)18.20
• Espnet-tts: Unified, Reproducible, And Integratable Open Source End-to-end Text-to-speech Toolkit (2019)23.32
• Espnet-spk: Full Pipeline Speaker Embedding Toolkit With Reproducible Recipes, Self-supervised Front-ends, And Off-the-shelf Models (2024)0.00
• EEND-SS: Joint End-to-end Neural Speaker Diarization And Speech Separation For Flexible Number Of Speakers (2022)10.35
• End-to-end Dereverberation, Beamforming, And Speech Recognition With Improved Numerical Stability And Advanced Frontend (2021)10.97
• Improving Voice Separation By Incorporating End-to-end Speech Recognition (2019)0.00