Improving Transformer-based Conversational ASR By Inter-sentential Attention Mechanism

Abstract

Transformer-based models have demonstrated their effectiveness in automatic speech recognition (ASR) tasks and even shown superior performance over the conventional hybrid framework. The main idea of Transformers is to capture the long-range global context within an utterance by self-attention layers. However, for scenarios like conversational speech, such utterance-level modeling will neglect contextual dependencies that span across utterances. In this paper, we propose to explicitly model the inter-sentential information in a Transformer based end-to-end architecture for conversational speech recognition. Specifically, for the encoder network, we capture the contexts of previous speech and incorporate such historic information into current input by a context-aware residual attention mechanism. For the decoder, the prediction of current utterance is also conditioned on the historic linguistic information through a conditional decoder framework. We show the effectiveness of our propose

Stats

• citations9
• S2 citations—
• github stars0
• HF likes0
• heat score7.50
• arxiv keywei2022improving

Related papers

• Transformer ASR With Contextual Block Processing (2019)0.00
• Transformers With Convolutional Context For ASR (2019)0.00
• Advanced Long-context End-to-end Speech Recognition Using Context-expanded Transformers (2021)10.07
• Attention-based ASR With Lightweight And Dynamic Convolutions (2019)9.03
• Towards Online End-to-end Transformer Automatic Speech Recognition (2019)0.00
• Linguistic-enhanced Transformer With CTC Embedding For Speech Recognition (2022)2.26
• Improving Hybrid Ctc/attention End-to-end Speech Recognition With Pretrained Acoustic And Language Model (2021)8.82
• Transformer-based Online Ctc/attention End-to-end Speech Recognition Architecture (2020)14.06