Dynamically Context-sensitive Time-decay Attention For Dialogue Modeling

Abstract

Spoken language understanding (SLU) is an essential component in conversational systems. Considering that contexts provide informative cues for better understanding, history can be leveraged for contextual SLU. However, most prior work only paid attention to the related content in history utterances and ignored the temporal information. In dialogues, it is intuitive that the most recent utterances are more important than the least recent ones, and time-aware attention should be in a decaying manner. Therefore, this paper allows the model to automatically learn a time-decay attention function where the attentional weights can be dynamically decided based on the content of each role's contexts, which effectively integrates both content-aware and time-aware perspectives and demonstrates remarkable flexibility to complex dialogue contexts. The experiments on the benchmark Dialogue State Tracking Challenge (DSTC4) dataset show that the proposed dynamically context-sensitive time-decay atten

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keysu2018dynamically

Related papers

• Decay-function-free Time-aware Attention To Context And Speaker Indicator For Spoken Language Understanding (2019)0.00
• Dynamic Time-aware Attention To Speaker Roles And Contexts For Spoken Language Understanding (2017)8.35
• Attentive Contextual Carryover For Multi-turn End-to-end Spoken Language Understanding (2021)7.16
• Attention-based Contextual Language Model Adaptation For Speech Recognition (2021)0.00
• Modeling Inter-speaker Relationship In Xlnet For Contextual Spoken Language Understanding (2019)0.00
• Game-time: Evaluating Temporal Dynamics In Spoken Language Models (2026)0.00
• The Speech-llm Takes It All: A Truly Fully End-to-end Spoken Dialogue State Tracking Approach (2025)0.00
• Improving End-to-end SLU Performance With Prosodic Attention And Distillation (2023)0.00