Federated Learning For Keyword Spotting

Abstract

We propose a practical approach based on federated learning to solve out-of-domain issues with continuously running embedded speech-based models such as wake word detectors. We conduct an extensive empirical study of the federated averaging algorithm for the "Hey Snips" wake word based on a crowdsourced dataset that mimics a federation of wake word users. We empirically demonstrate that using an adaptive averaging strategy inspired from Adam in place of standard weighted model averaging highly reduces the number of communication rounds required to reach our target performance. The associated upstream communication costs per user are estimated at 8 MB, which is a reasonable in the context of smart home voice assistants. Additionally, the dataset used for these experiments is being open sourced with the aim of fostering further transparent research in the application of federated learning to speech data.

Stats

• citations189
• S2 citations—
• github stars0
• HF likes0
• heat score17.09
• arxiv keyleroy2018federated

Related papers

• Fedspeech: Federated Text-to-speech With Continual Learning (2021)9.23
• Training Speech Recognition Models With Federated Learning: A Quality/cost Framework (2020)12.93
• The Gift Of Feedback: Improving ASR Model Quality By Learning From User Corrections Through Federated Learning (2023)0.00
• Communication-efficient Personalized Federated Learning For Speech-to-text Tasks (2024)7.81
• Boosting Keyword Spotting Through On-device Learnable User Speech Characteristics (2024)0.00
• Lightweight Feature Encoder For Wake-up Word Detection Based On Self-supervised Speech Representation (2023)5.84
• Optimizing Dysarthria Wake-up Word Spotting: An End-to-end Approach For SLT 2024 LRDWWS Challenge (2024)2.26
• Predicting Detection Filters For Small Footprint Open-vocabulary Keyword Spotting (2019)9.92