Private Language Model Adaptation For Speech Recognition

Abstract

Speech model adaptation is crucial to handle the discrepancy between server-side proxy training data and actual data received on local devices of users. With the use of federated learning (FL), we introduce an efficient approach on continuously adapting neural network language models (NNLMs) on private devices with applications on automatic speech recognition (ASR). To address the potential speech transcription errors in the on-device training corpus, we perform empirical studies on comparing various strategies of leveraging token-level confidence scores to improve the NNLM quality in the FL settings. Experiments show that compared with no model adaptation, the proposed method achieves relative 2.6% and 10.8% word error rate (WER) reductions on two speech evaluation datasets, respectively. We also provide analysis in evaluating privacy guarantees of our presented procedure.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyliu2021private

Related papers

• Federated Marginal Personalization For ASR Rescoring (2020)2.26
• Communication-efficient Personalized Federated Learning For Speech-to-text Tasks (2024)7.81
• The Gift Of Feedback: Improving ASR Model Quality By Learning From User Corrections Through Federated Learning (2023)0.00
• Fast Contextual Adaptation With Neural Associative Memory For On-device Personalized Speech Recognition (2021)9.76
• Privacy Attacks For Automatic Speech Recognition Acoustic Models In A Federated Learning Framework (2021)9.23
• Instance-based Model Adaptation For Direct Speech Translation (2019)0.00
• Enabling Differentially Private Federated Learning For Speech Recognition: Benchmarks, Adaptive Optimizers And Gradient Clipping (2023)2.56
• On-device Speaker Anonymization Of Acoustic Embeddings For ASR Based Onflexible Location Gradient Reversal Layer (2023)0.00