Do As I Mean, Not As I Say: Sequence Loss Training For Spoken Language Understanding
2021 · Milind Rao, Pranav Dheram, Gautam Tiwari, et al.
Abstract
Spoken language understanding (SLU) systems extract transcriptions, as well as semantics of intent or named entities from speech, and are essential components of voice activated systems. SLU models, which either directly extract semantics from audio or are composed of pipelined automatic speech recognition (ASR) and natural language understanding (NLU) models, are typically trained via differentiable cross-entropy losses, even when the relevant performance metrics of interest are word or semantic error rates. In this work, we propose non-differentiable sequence losses based on SLU metrics as a proxy for semantic error and use the REINFORCE trick to train ASR and SLU models with this loss. We show that custom sequence loss training is the state-of-the-art on open SLU datasets and leads to 6% relative improvement in both ASR and NLU performance metrics on large proprietary datasets. We also demonstrate how the semantic sequence loss training paradigm can be used to update ASR and SLU mod
Authors
(none)
Tags
Stats
Related papers
- On Joint Training With Interfaces For Spoken Language Understanding (2021)7.16
- Integrating Pretrained ASR And LM To Perform Sequence Generation For Spoken Language Understanding (2023)5.24
- Bridging The Gap Between Clean Data Training And Real-world Inference For Spoken Language Understanding (2021)0.00
- Building Robust Spoken Language Understanding By Cross Attention Between Phoneme Sequence And ASR Hypothesis (2022)2.26
- ML-LMCL: Mutual Learning And Large-margin Contrastive Learning For Improving ASR Robustness In Spoken Language Understanding (2023)0.00
- Token-level Sequence Labeling For Spoken Language Understanding Using Compositional End-to-end Models (2022)0.00
- Speech-language Pre-training For End-to-end Spoken Language Understanding (2021)9.41
- Recent Advances In End-to-end Spoken Language Understanding (2019)8.09