Wabert: A Low-resource End-to-end Model For Spoken Language Understanding And Speech-to-bert Alignment

Abstract

Historically lower-level tasks such as automatic speech recognition (ASR) and speaker identification are the main focus in the speech field. Interest has been growing in higher-level spoken language understanding (SLU) tasks recently, like sentiment analysis (SA). However, improving performances on SLU tasks remains a big challenge. Basically, there are two main methods for SLU tasks: (1) Two-stage method, which uses a speech model to transfer speech to text, then uses a language model to get the results of downstream tasks; (2) One-stage method, which just fine-tunes a pre-trained speech model to fit in the downstream tasks. The first method loses emotional cues such as intonation, and causes recognition errors during ASR process, and the second one lacks necessary language knowledge. In this paper, we propose the Wave BERT (WaBERT), a novel end-to-end model combining the speech model and the language model for SLU tasks. WaBERT is based on the pre-trained speech and language model, h

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyyao2022wabert

Related papers

• ST-BERT: Cross-modal Language Model Pre-training For End-to-end Spoken Language Understanding (2020)9.59
• Wav-bert: Cooperative Acoustic And Linguistic Representation Learning For Low-resource Speech Recognition (2021)8.82
• Large-scale Transfer Learning For Low-resource Spoken Language Understanding (2020)2.26
• Understanding Semantics From Speech Through Pre-training (2019)0.00
• Two-stage Textual Knowledge Distillation For End-to-end Spoken Language Understanding (2020)9.41
• End-to-end Spoken Language Understanding: Performance Analyses Of A Voice Command Task In A Low Resource Setting (2022)8.35
• Recent Advances In End-to-end Spoken Language Understanding (2019)8.09
• Speechbert: An Audio-and-text Jointly Learned Language Model For End-to-end Spoken Question Answering (2019)12.33