Fednst: Federated Noisy Student Training For Automatic Speech Recognition

Abstract

Federated Learning (FL) enables training state-of-the-art Automatic Speech Recognition (ASR) models on user devices (clients) in distributed systems, hence preventing transmission of raw user data to a central server. A key challenge facing practical adoption of FL for ASR is obtaining ground-truth labels on the clients. Existing approaches rely on clients to manually transcribe their speech, which is impractical for obtaining large training corpora. A promising alternative is using semi-/self-supervised learning approaches to leverage unlabelled user data. To this end, we propose FedNST, a novel method for training distributed ASR models using private and unlabelled user data. We explore various facets of FedNST, such as training models with different proportions of labelled and unlabelled data, and evaluate the proposed approach on 1173 simulated clients. Evaluating FedNST on LibriSpeech, where 960 hours of speech data is split equally into server (labelled) and client (unlabelled) d

Stats

• citations7
• S2 citations—
• github stars0
• HF likes0
• heat score6.77
• arxiv keymehmood2022fednst

Related papers

• Separate But Together: Unsupervised Federated Learning For Speech Enhancement From Non-iid Data (2021)8.35
• Communication-efficient Personalized Federated Learning For Speech-to-text Tasks (2024)7.81
• Fedspeech: Federated Text-to-speech With Continual Learning (2021)9.23
• Semi-fedser: Semi-supervised Learning For Speech Emotion Recognition On Federated Learning Using Multiview Pseudo-labeling (2022)8.82
• The Gift Of Feedback: Improving ASR Model Quality By Learning From User Corrections Through Federated Learning (2023)0.00
• Federated Pruning: Improving Neural Network Efficiency With Federated Learning (2022)7.50
• Training Speech Recognition Models With Federated Learning: A Quality/cost Framework (2020)12.93
• Private Language Model Adaptation For Speech Recognition (2021)0.00