A Pre-training Framework That Encodes Noise Information For Speech Quality Assessment

Abstract

Self-supervised learning (SSL) has grown in interest within the speech processing community, since it produces representations that are useful for many downstream tasks. SSL uses global and contextual methods to produce robust representations, where SSL even outperforms supervised models. Most self-supervised approaches, however, are limited to embedding information about, i.e., the phonemes, speaker identity, and emotion, into the extracted representations, where they become invariant to background sounds due to contrastive and auto-regressive learning. This is limiting because many downstream tasks leverage noise information to function accurately. Therefore, we propose a pre-training framework that learns information pertaining to background noise in a supervised manner, while jointly embedding speech information using a self-supervised strategy. We experiment with multiple encoders and show that our framework is useful for perceptual speech quality estimation, which relies on backg

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keysultana2024a

Related papers

• Non-contrastive Self-supervised Learning For Utterance-level Information Extraction From Speech (2022)9.59
• Feature Learning And Ensemble Pre-tasks Based Self-supervised Speech Denoising And Dereverberation (2022)0.00
• Efficient Extraction Of Noise-robust Discrete Units From Self-supervised Speech Models (2024)0.00
• Investigating Self-supervised Learning For Speech Enhancement And Separation (2022)13.44
• The Efficacy Of Self-supervised Speech Models For Audio Representations (2022)0.00
• Downstream Task Agnostic Speech Enhancement With Self-supervised Representation Loss (2023)6.77
• Progressive Residual Extraction Based Pre-training For Speech Representation Learning (2024)0.00
• Automatic Pronunciation Assessment Using Self-supervised Speech Representation Learning (2022)0.00