The Gift Of Feedback: Improving ASR Model Quality By Learning From User Corrections Through Federated Learning

Abstract

Automatic speech recognition (ASR) models are typically trained on large datasets of transcribed speech. As language evolves and new terms come into use, these models can become outdated and stale. In the context of models trained on the server but deployed on edge devices, errors may result from the mismatch between server training data and actual on-device usage. In this work, we seek to continually learn from on-device user corrections through Federated Learning (FL) to address this issue. We explore techniques to target fresh terms that the model has not previously encountered, learn long-tail words, and mitigate catastrophic forgetting. In experimental evaluations, we find that the proposed techniques improve model recognition of fresh terms, while preserving quality on the overall language distribution.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhou2023the

Related papers

• Importance Of Smoothness Induced By Optimizers In FL4ASR: Towards Understanding Federated Learning For End-to-end ASR (2023)0.00
• Private Language Model Adaptation For Speech Recognition (2021)0.00
• Federated Marginal Personalization For ASR Rescoring (2020)2.26
• Continual Learning For Monolingual End-to-end Automatic Speech Recognition (2021)7.16
• Fednst: Federated Noisy Student Training For Automatic Speech Recognition (2022)6.77
• Communication-efficient Personalized Federated Learning For Speech-to-text Tasks (2024)7.81
• Training Speech Recognition Models With Federated Learning: A Quality/cost Framework (2020)12.93
• Federated Learning For Keyword Spotting (2018)17.09