Large-scale Domain Adaptation Via Teacher-student Learning

Abstract

High accuracy speech recognition requires a large amount of transcribed data for supervised training. In the absence of such data, domain adaptation of a well-trained acoustic model can be performed, but even here, high accuracy usually requires significant labeled data from the target domain. In this work, we propose an approach to domain adaptation that does not require transcriptions but instead uses a corpus of unlabeled parallel data, consisting of pairs of samples from the source domain of the well-trained model and the desired target domain. To perform adaptation, we employ teacher/student (T/S) learning, in which the posterior probabilities generated by the source-domain model can be used in lieu of labels to train the target-domain model. We evaluate the proposed approach in two scenarios, adapting a clean acoustic model to noisy speech and adapting an adults speech acoustic model to children speech. Significant improvements in accuracy are obtained, with reductions in word er

Stats

• citations71
• S2 citations—
• github stars0
• HF likes0
• heat score13.93
• arxiv keyli2017large

Related papers

• Developing Far-field Speaker System Via Teacher-student Learning (2018)10.85
• Automatic Data Augmentation For Domain Adapted Fine-tuning Of Self-supervised Speech Representations (2023)0.00
• Advancing Multi-accented LSTM-CTC Speech Recognition Using A Domain Specific Student-teacher Learning Paradigm (2018)7.81
• Toward Domain-invariant Speech Recognition Via Large Scale Training (2018)13.39
• A Simple Baseline For Domain Adaptation In End To End ASR Systems Using Synthetic Data (2022)7.16
• Teach An All-rounder With Experts In Different Domains (2019)2.26
• Unsupervised Domain Adaptation For Speech Recognition With Unsupervised Error Correction (2022)5.24
• Examining Test-time Adaptation For Personalized Child Speech Recognition (2024)0.00