Confidence Score Based Conformer Speaker Adaptation For Speech Recognition

Abstract

A key challenge for automatic speech recognition (ASR) systems is to model the speaker level variability. In this paper, compact speaker dependent learning hidden unit contributions (LHUC) are used to facilitate both speaker adaptive training (SAT) and test time unsupervised speaker adaptation for state-of-the-art Conformer based end-to-end ASR systems. The sensitivity during adaptation to supervision error rate is reduced using confidence score based selection of the more "trustworthy" subset of speaker specific data. A confidence estimation module is used to smooth the over-confident Conformer decoder output probabilities before serving as confidence scores. The increased data sparsity due to speaker level data selection is addressed using Bayesian estimation of LHUC parameters. Experiments on the 300-hour Switchboard corpus suggest that the proposed LHUC-SAT Conformer with confidence score based test time unsupervised adaptation outperformed the baseline speaker independent and i-ve

Stats

• citations11
• S2 citations—
• github stars0
• HF likes0
• heat score8.09
• arxiv keydeng2022confidence

Related papers

• Confidence Score Based Speaker Adaptation Of Conformer Speech Recognition Systems (2023)8.35
• Unsupervised Model-based Speaker Adaptation Of End-to-end Lattice-free MMI Model For Speech Recognition (2022)2.26
• Learning Hidden Unit Contributions For Unsupervised Acoustic Model Adaptation (2016)14.47
• Factorised Speaker-environment Adaptive Training Of Conformer Speech Recognition Systems (2023)0.00
• Speaker Adaptive Training Using Model Agnostic Meta-learning (2019)9.92
• Bayesian Learning For Deep Neural Network Adaptation (2020)9.76
• Homogeneous Speaker Features For On-the-fly Dysarthric And Elderly Speaker Adaptation (2024)0.00
• On-the-fly Feature Based Rapid Speaker Adaptation For Dysarthric And Elderly Speech Recognition (2022)6.34