Knowledge-aware Audio-grounded Generative Slot Filling For Limited Annotated Data

Abstract

Manually annotating fine-grained slot-value labels for task-oriented dialogue (ToD) systems is an expensive and time-consuming endeavour. This motivates research into slot-filling methods that operate with limited amounts of labelled data. Moreover, the majority of current work on ToD is based solely on text as the input modality, neglecting the additional challenges of imperfect automatic speech recognition (ASR) when working with spoken language. In this work, we propose a Knowledge-Aware Audio-Grounded generative slot-filling framework, termed KA2G, that focuses on few-shot and zero-shot slot filling for ToD with speech input. KA2G achieves robust and data-efficient slot filling for speech-based ToD by 1) framing it as a text generation task, 2) grounding text generation additionally in the audio modality, and 3) conditioning on available external knowledge (e.g. a predefined list of possible slot values). We show that combining both modalities within the KA2G framework improves the

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keysun2023knowledge

Related papers

• C2c-genda: Cluster-to-cluster Generation For Data Augmentation Of Slot Filling (2020)7.16
• Multi-domain Adversarial Learning For Slot Filling In Spoken Language Understanding (2017)0.00
• Improving Slot Filling By Utilizing Contextual Information (2019)2.26
• Leave No Knowledge Behind During Knowledge Distillation: Towards Practical And Effective Knowledge Distillation For Code-switching ASR Using Realistic Data (2024)3.58
• Latent Filling: Latent Space Data Augmentation For Zero-shot Speech Synthesis (2023)4.52
• Data Augmentation For Spoken Language Understanding Via Joint Variational Generation (2018)10.61
• Joint Learning Of Word And Label Embeddings For Sequence Labelling In Spoken Language Understanding (2019)3.58
• MT2KD: Towards A General-purpose Encoder For Speech, Speaker, And Audio Events (2024)0.00