Accurate And Reliable Confidence Estimation Based On Non-autoregressive End-to-end Speech Recognition System
2023 · Xian Shi, Haoneng Luo, Zhifu Gao, et al.
Abstract
Estimating confidence scores for recognition results is a classic task in ASR field and of vital importance for kinds of downstream tasks and training strategies. Previous end-to-end~(E2E) based confidence estimation models (CEM) predict score sequences of equal length with input transcriptions, leading to unreliable estimation when deletion and insertion errors occur. In this paper we proposed CIF-Aligned confidence estimation model (CA-CEM) to achieve accurate and reliable confidence estimation based on novel non-autoregressive E2E ASR model - Paraformer. CA-CEM utilizes the modeling character of continuous integrate-and-fire (CIF) mechanism to generate token-synchronous acoustic embedding, which solves the estimation failure issue above. We measure the quality of estimation with AUC and RMSE in token level and ECE-U - a proposed metrics in utterance level. CA-CEM gains 24% and 19% relative reduction on ECE-U and also better AUC and RMSE on two test sets. Furthermore, we conduct anal
Authors
(none)
Tags
Stats
Related papers
- Confidence Estimation For Attention-based Sequence-to-sequence Models For Speech Recognition (2020)11.49
- An Evaluation Of Word-level Confidence Estimation For End-to-end Automatic Speech Recognition (2021)0.00
- Teles: Temporal Lexeme Similarity Score To Estimate Confidence In End-to-end ASR (2024)6.34
- Utterance-level Neural Confidence Measure For End-to-end Children Speech Recognition (2021)6.77
- Fast Entropy-based Methods Of Word-level Confidence Estimation For End-to-end Automatic Speech Recognition (2022)7.16
- Semantic-aware Confidence Calibration For Automated Audio Captioning (2025)0.00
- Sequence-level Confidence Classifier For ASR Utterance Accuracy And Application To Acoustic Models (2021)5.24
- Multi-task Learning For End-to-end ASR Word And Utterance Confidence With Deletion Prediction (2021)7.50