Unsupervised Model-based Speaker Adaptation Of End-to-end Lattice-free MMI Model For Speech Recognition

Abstract

Modeling the speaker variability is a key challenge for automatic speech recognition (ASR) systems. In this paper, the learning hidden unit contributions (LHUC) based adaptation techniques with compact speaker dependent (SD) parameters are used to facilitate both speaker adaptive training (SAT) and unsupervised test-time speaker adaptation for end-to-end (E2E) lattice-free MMI (LF-MMI) models. An unsupervised model-based adaptation framework is proposed to estimate the SD parameters in E2E paradigm using LF-MMI and cross entropy (CE) criterions. Various regularization methods of the standard LHUC adaptation, e.g., the Bayesian LHUC (BLHUC) adaptation, are systematically investigated to mitigate the risk of overfitting, on E2E LF-MMI CNN-TDNN and CNN-TDNN-BLSTM models. Lattice-based confidence score estimation is used for adaptation data selection to reduce the supervision label uncertainty. Experiments on the 300-hour Switchboard task suggest that applying BLHUC in the proposed unsuper

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keyxie2022unsupervised

Related papers

• Learning Hidden Unit Contributions For Unsupervised Acoustic Model Adaptation (2016)14.47
• Confidence Score Based Conformer Speaker Adaptation For Speech Recognition (2022)8.09
• Bayesian Learning For Deep Neural Network Adaptation (2020)9.76
• Learning To Adapt: A Meta-learning Approach For Speaker Adaptation (2018)9.76
• Consistent Training And Decoding For End-to-end Speech Recognition Using Lattice-free MMI (2021)8.35
• Speaker Adaptation For End-to-end CTC Models (2019)8.60
• Empirical Evaluation Of Speaker Adaptation On DNN Based Acoustic Model (2018)5.24
• Confidence Score Based Speaker Adaptation Of Conformer Speech Recognition Systems (2023)8.35