Feature Learning And Ensemble Pre-tasks Based Self-supervised Speech Denoising And Dereverberation

Abstract

Self-supervised learning (SSL) achieves great success in monaural speech enhancement, while the accuracy of the target speech estimation, particularly for unseen speakers, remains inadequate with existing pre-tasks. As speech signal contains multi-faceted information including speaker identity, paralinguistics, and spoken content, the latent representation for speech enhancement becomes a tough task. In this paper, we study the effectiveness of each feature which is commonly used in speech enhancement and exploit the feature combination in the SSL case. Besides, we propose an ensemble training strategy. The latent representation of the clean speech signal is learned, meanwhile, the dereverberated mask and the estimated ratio mask are exploited to denoise and dereverberate the mixture. The latent representation learning and the masks estimation are considered as two pre-tasks in the training stage. In addition, to study the effectiveness between the pre-tasks, we compare different train

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyli2022feature

Related papers

• Investigating Self-supervised Learning For Speech Enhancement And Separation (2022)13.44
• Self-supervised Learning Based Monaural Speech Enhancement With Multi-task Pre-training (2021)0.00
• Investigation Of Ensemble Features Of Self-supervised Pretrained Models For Automatic Speech Recognition (2022)9.41
• The Efficacy Of Self-supervised Speech Models For Audio Representations (2022)0.00
• Downstream Task Agnostic Speech Enhancement With Self-supervised Representation Loss (2023)6.77
• A Pre-training Framework That Encodes Noise Information For Speech Quality Assessment (2024)3.58
• Weakly-supervised Speech Pre-training: A Case Study On Target Speech Recognition (2023)8.09
• Unispeech-sat: Universal Speech Representation Learning With Speaker Aware Pre-training (2021)0.00