Learning Hidden Unit Contributions For Unsupervised Acoustic Model Adaptation

Abstract

This work presents a broad study on the adaptation of neural network acoustic models by means of learning hidden unit contributions (LHUC) -- a method that linearly re-combines hidden units in a speaker- or environment-dependent manner using small amounts of unsupervised adaptation data. We also extend LHUC to a speaker adaptive training (SAT) framework that leads to a more adaptable DNN acoustic model, working both in a speaker-dependent and a speaker-independent manner, without the requirements to maintain auxiliary speaker-dependent feature extractors or to introduce significant speaker-dependent changes to the DNN structure. Through a series of experiments on four different speech recognition benchmarks (TED talks, Switchboard, AMI meetings, and Aurora4) comprising 270 test speakers, we show that LHUC in both its test-only and SAT variants results in consistent word error rate reductions ranging from 5% to 23% relative depending on the task and the degree of mismatch between traini

Stats

• citations84
• S2 citations—
• github stars0
• HF likes0
• heat score14.47
• arxiv keyswietojanski2016learning

Related papers

• Unsupervised Model-based Speaker Adaptation Of End-to-end Lattice-free MMI Model For Speech Recognition (2022)2.26
• Learning To Adapt: A Meta-learning Approach For Speaker Adaptation (2018)9.76
• Confidence Score Based Conformer Speaker Adaptation For Speech Recognition (2022)8.09
• Speaker Adaptive Training Using Model Agnostic Meta-learning (2019)9.92
• Empirical Evaluation Of Speaker Adaptation On DNN Based Acoustic Model (2018)5.24
• Bayesian Learning For Deep Neural Network Adaptation (2020)9.76
• Learning Separable Hidden Unit Contributions For Speaker-adaptive Lip-reading (2023)0.00
• Unsupervised Accent Adaptation Through Masked Language Model Correction Of Discrete Self-supervised Speech Units (2023)4.52