Instant One-shot Word-learning For Context-specific Neural Sequence-to-sequence Speech Recognition

Abstract

Neural sequence-to-sequence systems deliver state-of-the-art performance for automatic speech recognition (ASR). When using appropriate modeling units, e.g., byte-pair encoded characters, these systems are in principal open vocabulary systems. In practice, however, they often fail to recognize words not seen during training, e.g., named entities, numbers or technical terms. To alleviate this problem we supplement an end-to-end ASR system with a word/phrase memory and a mechanism to access this memory to recognize the words and phrases correctly. After the training of the ASR system, and when it has already been deployed, a relevant word can be added or subtracted instantly without the need for further training. In this paper we demonstrate that through this mechanism our system is able to recognize more than 85% of newly added words that it previously failed to recognize compared to a strong baseline.

Stats

• citations18
• S2 citations—
• github stars0
• HF likes0
• heat score9.59
• arxiv keyhuber2021instant

Related papers

• Fast Contextual Adaptation With Neural Associative Memory For On-device Personalized Speech Recognition (2021)9.76
• Phoneme Based Neural Transducer For Large Vocabulary Speech Recognition (2020)9.59
• Acoustic-to-word Recognition With Sequence-to-sequence Models (2018)6.77
• Deep Context: End-to-end Contextual Speech Recognition (2018)15.57
• Improving Neural Biasing For Contextual Speech Recognition By Early Context Injection And Text Perturbation (2024)8.09
• State-of-the-art Speech Recognition With Sequence-to-sequence Models (2017)21.01
• Using Synthetic Audio To Improve The Recognition Of Out-of-vocabulary Words In End-to-end ASR Systems (2020)12.33
• PROCTER: Pronunciation-aware Contextual Adapter For Personalized Speech Recognition In Neural Transducers (2023)8.60