Separate But Together: Unsupervised Federated Learning For Speech Enhancement From Non-iid Data

Abstract

We propose FEDENHANCE, an unsupervised federated learning (FL) approach for speech enhancement and separation with non-IID distributed data across multiple clients. We simulate a real-world scenario where each client only has access to a few noisy recordings from a limited and disjoint number of speakers (hence non-IID). Each client trains their model in isolation using mixture invariant training while periodically providing updates to a central server. Our experiments show that our approach achieves competitive enhancement performance compared to IID training on a single device and that we can further facilitate the convergence speed and the overall performance using transfer learning on the server-side. Moreover, we show that we can effectively combine updates from clients trained locally with supervised and unsupervised losses. We also release a new dataset LibriFSD50K and its creation recipe in order to facilitate FL research for source separation problems.

Stats

• citations12
• S2 citations—
• github stars0
• HF likes0
• heat score8.35
• arxiv keytzinis2021separate

Related papers

• Fednst: Federated Noisy Student Training For Automatic Speech Recognition (2022)6.77
• Semi-fedser: Semi-supervised Learning For Speech Emotion Recognition On Federated Learning Using Multiview Pseudo-labeling (2022)8.82
• Communication-efficient Personalized Federated Learning For Speech-to-text Tasks (2024)7.81
• Training Speech Recognition Models With Federated Learning: A Quality/cost Framework (2020)12.93
• Fedspeech: Federated Text-to-speech With Continual Learning (2021)9.23
• Investigating Self-supervised Learning For Speech Enhancement And Separation (2022)13.44
• Unsupervised Speaker Diarization In Distributed Iot Networks Using Federated Learning (2024)4.52
• Heterogeneous Space Fusion And Dual-dimension Attention: A New Paradigm For Speech Enhancement (2024)2.26