I\(^2\)KD-SLU: An Intra-inter Knowledge Distillation Framework For Zero-shot Cross-lingual Spoken Language Understanding
2023 · Tianjun Mao, Chenghong Zhang
Abstract
Spoken language understanding (SLU) typically includes two subtasks: intent detection and slot filling. Currently, it has achieved great success in high-resource languages, but it still remains challenging in low-resource languages due to the scarcity of labeled training data. Hence, there is a growing interest in zero-shot cross-lingual SLU. Despite of the success of existing zero-shot cross-lingual SLU models, most of them neglect to achieve the mutual guidance between intent and slots. To address this issue, we propose an Intra-Inter Knowledge Distillation framework for zero-shot cross-lingual Spoken Language Understanding (I\(^2\)KD-SLU) to model the mutual guidance. Specifically, we not only apply intra-knowledge distillation between intent predictions or slot predictions of the same utterance in different languages, but also apply inter-knowledge distillation between intent predictions and slot predictions of the same utterance. Our experimental results demonstrate that our propo
Authors
(none)
Tags
Stats
Related papers
- Two-stage Textual Knowledge Distillation For End-to-end Spoken Language Understanding (2020)9.41
- AFD-SLU: Adaptive Feature Distillation For Spoken Language Understanding (2025)0.00
- Integrated Multi-level Knowledge Distillation For Enhanced Speaker Verification (2024)0.00
- Gl-clef: A Global-local Contrastive Learning Framework For Cross-lingual Spoken Language Understanding (2022)10.35
- BLSP-KD: Bootstrapping Language-speech Pre-training Via Knowledge Distillation (2024)0.00
- Sequence-level Knowledge Distillation For Class-incremental End-to-end Spoken Language Understanding (2023)0.00
- HC\(^2\)L: Hybrid And Cooperative Contrastive Learning For Cross-lingual Spoken Language Understanding (2024)4.52
- A Novel Bi-directional Interrelated Model For Joint Intent Detection And Slot Filling (2019)12.33