ASR-GLUE: A New Multi-task Benchmark For Asr-robust Natural Language Understanding
2021 · Lingyun Feng, Jianwei Yu, Deng Cai, et al.
Abstract
Language understanding in speech-based systems have attracted much attention in recent years with the growing demand for voice interface applications. However, the robustness of natural language understanding (NLU) systems to errors introduced by automatic speech recognition (ASR) is under-examined. %To facilitate the research on ASR-robust general language understanding, In this paper, we propose ASR-GLUE benchmark, a new collection of 6 different NLU tasks for evaluating the performance of models under ASR error across 3 different levels of background noise and 6 speakers with various voice characteristics. Based on the proposed benchmark, we systematically investigate the effect of ASR error on NLU tasks in terms of noise intensity, error type and speaker variants. We further purpose two ways, correction-based method and data augmentation-based method to improve robustness of the NLU systems. Extensive experimental results and analysises show that the proposed methods are effective
Authors
(none)
Tags
Stats
Related papers
- Contrastive Learning For Improving ASR Robustness In Spoken Language Understanding (2022)6.34
- ML-LMCL: Mutual Learning And Large-margin Contrastive Learning For Improving ASR Robustness In Spoken Language Understanding (2023)0.00
- Building Robust Spoken Language Understanding By Cross Attention Between Phoneme Sequence And ASR Hypothesis (2022)2.26
- Speechcolab Leaderboard: An Open-source Platform For Automatic Speech Recognition Evaluation (2024)9.05
- ASR Error Management For Improving Spoken Language Understanding (2017)9.92
- Towards ASR Robust Spoken Language Understanding Through In-context Learning With Word Confusion Networks (2024)0.00
- SLUE Phase-2: A Benchmark Suite Of Diverse Spoken Language Understanding Tasks (2022)10.07
- Multimodal Audio-textual Architecture For Robust Spoken Language Understanding (2023)0.00