Continuously Learning New Words In Automatic Speech Recognition

Abstract

Despite recent advances, Automatic Speech Recognition (ASR) systems are still far from perfect. Typical errors include acronyms, named entities, and domain-specific special words for which little or no labeled data is available. To address the problem of recognizing these words, we propose a self-supervised continual learning approach: Given the audio of a lecture talk with the corresponding slides, we bias the model towards decoding new words from the slides by using a memory-enhanced ASR model from the literature. Then, we perform inference on the talk, collecting utterances that contain detected new words into an adaptation data set. Continual learning is then performed by training adaptation weights added to the model on this data set. The whole procedure is iterated for many talks. We show that with this approach, we obtain increasing performance on the new words when they occur more frequently (more than 80% recall) while preserving the general performance of the model.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyhuber2024continuously

Related papers

• Continual Learning For Monolingual End-to-end Automatic Speech Recognition (2021)7.16
• Unsupervised Online Continual Learning For Automatic Speech Recognition (2024)4.52
• Rehearsal-free Online Continual Learning For Automatic Speech Recognition (2023)5.24
• Continual Learning Optimizations For Auto-regressive Decoder Of Multilingual ASR Systems (2024)5.84
• Continual Speaker Adaptation For Text-to-speech Synthesis (2021)0.00
• Continual Learning In Machine Speech Chain Using Gradient Episodic Memory (2024)2.26
• Self-taught Recognizer: Toward Unsupervised Adaptation For Speech Foundation Models (2024)2.26
• Instant One-shot Word-learning For Context-specific Neural Sequence-to-sequence Speech Recognition (2021)9.59