Contrastive Learning For Improving ASR Robustness In Spoken Language Understanding

Abstract

Spoken language understanding (SLU) is an essential task for machines to understand human speech for better interactions. However, errors from the automatic speech recognizer (ASR) usually hurt the understanding performance. In reality, ASR systems may not be easy to adjust for the target scenarios. Therefore, this paper focuses on learning utterance representations that are robust to ASR errors using a contrastive objective, and further strengthens the generalization ability by combining supervised contrastive learning and self-distillation in model fine-tuning. Experiments on three benchmark datasets demonstrate the effectiveness of our proposed approach.

Stats

• citations6
• S2 citations—
• github stars0
• HF likes0
• heat score6.34
• arxiv keychang2022contrastive

Related papers

• ML-LMCL: Mutual Learning And Large-margin Contrastive Learning For Improving ASR Robustness In Spoken Language Understanding (2023)0.00
• Building Robust Spoken Language Understanding By Cross Attention Between Phoneme Sequence And ASR Hypothesis (2022)2.26
• Towards ASR Robust Spoken Language Understanding Through In-context Learning With Word Confusion Networks (2024)0.00
• Modality Confidence Aware Training For Robust End-to-end Spoken Language Understanding (2023)2.26
• Learning Asr-robust Contextualized Embeddings For Spoken Language Understanding (2019)12.02
• Bridging The Gap Between Clean Data Training And Real-world Inference For Spoken Language Understanding (2021)0.00
• ASR-GLUE: A New Multi-task Benchmark For Asr-robust Natural Language Understanding (2021)0.00
• A Study On The Integration Of Pre-trained SSL, ASR, LM And SLU Models For Spoken Language Understanding (2022)8.09