End-to-end Integration Of Speech Recognition, Dereverberation, Beamforming, And Self-supervised Learning Representation

Abstract

Self-supervised learning representation (SSLR) has demonstrated its significant effectiveness in automatic speech recognition (ASR), mainly with clean speech. Recent work pointed out the strength of integrating SSLR with single-channel speech enhancement for ASR in noisy environments. This paper further advances this integration by dealing with multi-channel input. We propose a novel end-to-end architecture by integrating dereverberation, beamforming, SSLR, and ASR within a single neural network. Our system achieves the best performance reported in the literature on the CHiME-4 6-channel track with a word error rate (WER) of 1.77%. While the WavLM-based strong SSLR demonstrates promising results by itself, the end-to-end integration with the weighted power minimization distortionless response beamformer, which simultaneously performs dereverberation and denoising, improves WER significantly. Its effectiveness is also validated on the REVERB dataset.

Stats

• citations13
• S2 citations—
• github stars0
• HF likes0
• heat score8.60
• arxiv keymasuyama2022end

Related papers

• Exploring The Integration Of Speech Separation And Recognition With Self-supervised Learning Representation (2023)6.34
• End-to-end Integration Of Speech Recognition, Speech Enhancement, And Self-supervised Learning Representation (2022)12.54
• End-to-end Dereverberation, Beamforming, And Speech Recognition With Improved Numerical Stability And Advanced Frontend (2021)10.97
• End-to-end Far-field Speech Recognition With Unified Dereverberation And Beamforming (2020)10.61
• Fearless: Feature Refinement Loss For Ensembling Self-supervised Learning Features In Robust End-to-end Speech Recognition (2022)6.77
• Deep Long Short-term Memory Adaptive Beamforming Networks For Multichannel Robust Speech Recognition (2017)13.23
• SSHR: Leveraging Self-supervised Hierarchical Representations For Multilingual Automatic Speech Recognition (2023)0.00
• An Investigation Of End-to-end Multichannel Speech Recognition For Reverberant And Mismatch Conditions (2019)0.00