AFD-SLU: Adaptive Feature Distillation For Spoken Language Understanding

Abstract

Spoken Language Understanding (SLU) is a core component of conversational systems, enabling machines to interpret user utterances. Despite its importance, developing effective SLU systems remains challenging due to the scarcity of labeled training data and the computational burden of deploying Large Language Models (LLMs) in real-world applications. To further alleviate these issues, we propose an Adaptive Feature Distillation framework that transfers rich semantic representations from a General Text Embeddings (GTE)-based teacher model to a lightweight student model. Our method introduces a dynamic adapter equipped with a Residual Projection Neural Network (RPNN) to align heterogeneous feature spaces, and a Dynamic Distillation Coefficient (DDC) that adaptively modulates the distillation strength based on real-time feedback from intent and slot prediction performance. Experiments on the Chinese profile-based ProSLU benchmark demonstrate that AFD-SLU achieves state-of-the-art results,

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyxie2025afd

Related papers

• Bridging The Gap Between Clean Data Training And Real-world Inference For Spoken Language Understanding (2021)0.00
• I\(^2\)KD-SLU: An Intra-inter Knowledge Distillation Framework For Zero-shot Cross-lingual Spoken Language Understanding (2023)0.00
• Two-stage Textual Knowledge Distillation For End-to-end Spoken Language Understanding (2020)9.41
• Discreteslu: A Large Language Model With Self-supervised Discrete Speech Units For Spoken Language Understanding (2024)5.84
• Unislu: Unified Spoken Language Understanding From Heterogeneous Cross-task Datasets (2025)0.00
• Improving End-to-end SLU Performance With Prosodic Attention And Distillation (2023)0.00
• Exploring Fine-tuning Of Large Audio Language Models For Spoken Language Understanding Under Limited Speech Data (2025)0.00
• Large-scale Transfer Learning For Low-resource Spoken Language Understanding (2020)2.26