Fast Word Error Rate Estimation Using Self-supervised Representations For Speech And Text
2023 · Chanho Park, Chengsong Lu, Mingjie Chen, et al.
Abstract
Word error rate (WER) estimation aims to evaluate the quality of an automatic speech recognition (ASR) system's output without requiring ground-truth labels. This task has gained increasing attention as advanced ASR systems are trained on large amounts of data. In this context, the computational efficiency of a WER estimator becomes essential in practice. However, previous works have not prioritised this aspect. In this paper, a Fast estimator for WER (Fe-WER) is introduced, utilizing average pooling over self-supervised learning representations for speech and text. Our results demonstrate that Fe-WER outperformed a baseline relatively by 14.10% in root mean square error and 1.22% in Pearson correlation coefficient on Ted-Lium3. Moreover, a comparative analysis of the distributions of target WER and WER estimates was conducted, including an examination of the average values per speaker. Lastly, the inference speed was approximately 3.4 times faster in the real-time factor.
Authors
(none)
Tags
Stats
Related papers
- Automatic Speech Recognition System-independent Word Error Rate Estimation (2024)3.58
- Beyond Levenshtein: Leveraging Multiple Algorithms For Robust Word Error Rate Computations And Granular Error Classifications (2024)2.26
- On Word Error Rate Definitions And Their Efficient Computation For Multi-speaker Speech Recognition Systems (2022)9.76
- Predicting Word Error Rate For Reverberant Speech (2019)7.16
- Semantic-wer: A Unified Metric For The Evaluation Of ASR Transcript For End Usability (2021)0.00
- WER-BERT: Automatic WER Estimation With BERT In A Balanced Ordinal Classification Paradigm (2021)0.00
- On The Impact Of Word Error Rate On Acoustic-linguistic Speech Emotion Recognition: An Update For The Deep Learning Era (2021)0.00
- Lightweight Feature Encoder For Wake-up Word Detection Based On Self-supervised Speech Representation (2023)5.84