Improving Self-supervised Learning For Audio Representations By Feature Diversity And Decorrelation

Abstract

Self-supervised learning (SSL) has recently shown remarkable results in closing the gap between supervised and unsupervised learning. The idea is to learn robust features that are invariant to distortions of the input data. Despite its success, this idea can suffer from a collapsing issue where the network produces a constant representation. To this end, we introduce SELFIE, a novel Self-supervised Learning approach for audio representation via Feature Diversity and Decorrelation. SELFIE avoids the collapsing issue by ensuring that the representation (i) maintains a high diversity among embeddings and (ii) decorrelates the dependencies between dimensions. SELFIE is pre-trained on the large-scale AudioSet dataset and its embeddings are validated on nine audio downstream tasks, including speech, music, and sound event recognition. Experimental results show that SELFIE outperforms existing SSL methods in several tasks.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keynguyen2023improving

Related papers

• The Efficacy Of Self-supervised Speech Models For Audio Representations (2022)0.00
• Investigating Self-supervised Learning For Speech Enhancement And Separation (2022)13.44
• Self-supervised Learning Of Audio Representations Using Angular Contrastive Loss (2022)5.84
• Deploying Self-supervised Learning In The Wild For Hybrid Automatic Speech Recognition (2022)0.00
• Feature Learning And Ensemble Pre-tasks Based Self-supervised Speech Denoising And Dereverberation (2022)0.00
• Downstream Task Agnostic Speech Enhancement With Self-supervised Representation Loss (2023)6.77
• Understanding Self-supervised Learning Of Speech Representation Via Invariance And Redundancy Reduction (2023)0.00
• SLICER: Learning Universal Audio Representations Using Low-resource Self-supervised Pre-training (2022)0.00