Self-supervised Learning From Contrastive Mixtures For Personalized Speech Enhancement

Abstract

This work explores how self-supervised learning can be universally used to discover speaker-specific features towards enabling personalized speech enhancement models. We specifically address the few-shot learning scenario where access to cleaning recordings of a test-time speaker is limited to a few seconds, but noisy recordings of the speaker are abundant. We develop a simple contrastive learning procedure which treats the abundant noisy data as makeshift training targets through pairwise noise injection: the model is pretrained to maximize agreement between pairs of differently deformed identical utterances and to minimize agreement between pairs of similarly deformed nonidentical utterances. Our experiments compare the proposed pretraining approach with two baseline alternatives: speaker-agnostic fully-supervised pretraining, and speaker-specific self-supervised pretraining without contrastive loss terms. Of all three approaches, the proposed method using contrastive mixtures is fou

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keysivaraman2020self

Related papers

• Efficient Personalized Speech Enhancement Through Self-supervised Learning (2021)10.21
• Personalized Speech Enhancement Through Self-supervised Data Augmentation And Purification (2021)9.92
• Automatic Data Augmentation Selection And Parametrization In Contrastive Self-supervised Speech Representation Learning (2022)5.24
• Self-supervised Speaker Recognition Training Using Human-machine Dialogues (2022)5.84
• Zero-shot Personalized Speech Enhancement Through Speaker-informed Model Selection (2021)7.16
• Continual Self-training With Bootstrapped Remixing For Speech Enhancement (2021)7.81
• Label-efficient Self-supervised Speaker Verification With Information Maximization And Contrastive Learning (2022)6.77
• Self-supervised Training Of Speaker Encoder With Multi-modal Diverse Positive Pairs (2022)8.35