Towards Reducing The Need For Speech Training Data To Build Spoken Language Understanding Systems

Abstract

The lack of speech data annotated with labels required for spoken language understanding (SLU) is often a major hurdle in building end-to-end (E2E) systems that can directly process speech inputs. In contrast, large amounts of text data with suitable labels are usually available. In this paper, we propose a novel text representation and training methodology that allows E2E SLU systems to be effectively constructed using these text resources. With very limited amounts of additional speech, we show that these models can be further improved to perform at levels close to similar systems built on the full speech datasets. The efficacy of our proposed approach is demonstrated on both intent and entity tasks using three different SLU datasets. With text-only training, the proposed system achieves up to 90% of the performance possible with full speech training. With just an additional 10% of speech data, these models significantly improve further to 97% of full performance.

Stats

• citations12
• S2 citations—
• github stars0
• HF likes0
• heat score8.35
• arxiv keythomas2022towards

Related papers

• Using Speech Synthesis To Train End-to-end Spoken Language Understanding Models (2019)9.23
• Leveraging Unpaired Text Data For Training End-to-end Speech-to-intent Systems (2020)0.00
• Improving End-to-end Speech Processing By Efficient Text Data Utilization With Latent Synthesis (2023)0.00
• Large-scale Transfer Learning For Low-resource Spoken Language Understanding (2020)2.26
• Data Augmentation For Spoken Language Understanding Via Pretrained Language Models (2020)0.00
• Speech-language Pre-training For End-to-end Spoken Language Understanding (2021)9.41
• Pre-training For Spoken Language Understanding With Joint Textual And Phonetic Representation Learning (2021)2.26
• Modality Confidence Aware Training For Robust End-to-end Spoken Language Understanding (2023)2.26