Exploiting Consistency-preserving Loss And Perceptual Contrast Stretching To Boost Ssl-based Speech Enhancement

Abstract

Self-supervised representation learning (SSL) has attained SOTA results on several downstream speech tasks, but SSL-based speech enhancement (SE) solutions still lag behind. To address this issue, we exploit three main ideas: (i) Transformer-based masking generation, (ii) consistency-preserving loss, and (iii) perceptual contrast stretching (PCS). In detail, conformer layers, leveraging an attention mechanism, are introduced to effectively model frame-level representations and obtain the Ideal Ratio Mask (IRM) for SE. Moreover, we incorporate consistency in the loss function, which processes the input to account for the inconsistency effects of signal reconstruction from the spectrogram. Finally, PCS is employed to improve the contrast of input and target features according to perceptual importance. Evaluated on the VoiceBank-DEMAND task, the proposed solution outperforms previously SSL-based SE solutions when tested on several objective metrics, attaining a SOTA PESQ score of 3.54.

Stats

• citations7
• S2 citations—
• github stars0
• HF likes0
• heat score6.77
• arxiv keykhan2024exploiting

Related papers

• Downstream Task Agnostic Speech Enhancement With Self-supervised Representation Loss (2023)6.77
• Investigating Self-supervised Learning For Speech Enhancement And Separation (2022)13.44
• Magnitude-phase Dual-path Speech Enhancement Network Based On Self-supervised Embedding And Perceptual Contrast Stretch Boosting (2025)3.21
• Target Speech Extraction With Pre-trained Self-supervised Learning Models (2024)9.41
• Recycle-and-distill: Universal Compression Strategy For Transformer-based Speech SSL Models With Attention Map Reusing And Masking Distillation (2023)5.84
• Low-resource Self-supervised Learning With Ssl-enhanced TTS (2023)0.00
• A Closer Look At Wav2vec2 Embeddings For On-device Single-channel Speech Enhancement (2024)0.00
• Exploring Self-supervised Multi-view Contrastive Learning For Speech Emotion Recognition With Limited Annotations (2024)3.58