Mitigating The Impact Of Speech Recognition Errors On Spoken Question Answering By Adversarial Domain Adaptation

Abstract

Spoken question answering (SQA) is challenging due to complex reasoning on top of the spoken documents. The recent studies have also shown the catastrophic impact of automatic speech recognition (ASR) errors on SQA. Therefore, this work proposes to mitigate the ASR errors by aligning the mismatch between ASR hypotheses and their corresponding reference transcriptions. An adversarial model is applied to this domain adaptation task, which forces the model to learn domain-invariant features the QA model can effectively utilize in order to improve the SQA results. The experiments successfully demonstrate the effectiveness of our proposed model, and the results are better than the previous best model by 2% EM score.

Stats

• citations7
• S2 citations—
• github stars0
• HF likes0
• heat score6.77
• arxiv keylee2019mitigating

Related papers

• Self-supervised Contrastive Cross-modality Representation Learning For Spoken Question Answering (2021)9.41
• Mitigating The Impact Of Speech Recognition Errors On Chatbot Using Sequence-to-sequence Model (2017)7.81
• Speechbert: An Audio-and-text Jointly Learned Language Model For End-to-end Spoken Question Answering (2019)12.33
• Speaker Verification Using End-to-end Adversarial Language Adaptation (2018)11.19
• Unsupervised Domain Adaptation For Speech Recognition With Unsupervised Error Correction (2022)5.24
• QFA2SR: Query-free Adversarial Transfer Attacks To Speaker Recognition Systems (2023)0.00
• Enhancing And Adversarial: Improve ASR With Speaker Labels (2022)5.24
• Exploring Textual And Speech Information In Dialogue Act Classification With Speaker Domain Adaptation (2018)0.00