Effective Cross-utterance Language Modeling For Conversational Speech Recognition

Abstract

Conversational speech normally is embodied with loose syntactic structures at the utterance level but simultaneously exhibits topical coherence relations across consecutive utterances. Prior work has shown that capturing longer context information with a recurrent neural network or long short-term memory language model (LM) may suffer from the recent bias while excluding the long-range context. In order to capture the long-term semantic interactions among words and across utterances, we put forward disparate conversation history fusion methods for language modeling in automatic speech recognition (ASR) of conversational speech. Furthermore, a novel audio-fusion mechanism is introduced, which manages to fuse and utilize the acoustic embeddings of a current utterance and the semantic content of its corresponding conversation history in a cooperative way. To flesh out our ideas, we frame the ASR N-best hypothesis rescoring task as a prediction problem, leveraging BERT, an iconic pre-train

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keyyan2021effective

Related papers

• Leveraging Acoustic Contextual Representation By Audio-textual Cross-modal Learning For Conversational ASR (2022)0.00
• Contextual Biasing Of Language Models For Speech Recognition In Goal-oriented Conversational Agents (2021)0.00
• Context And System Fusion In Post-asr Emotion Recognition With Large Language Models (2024)0.00
• Cross-utterance ASR Rescoring With Graph-based Label Propagation (2023)2.26
• Transformer Language Models With Lstm-based Cross-utterance Information Representation (2021)10.48
• Learning Cross-lingual Mappings For Data Augmentation To Improve Low-resource Speech Recognition (2023)0.00
• Improving Transformer-based Conversational ASR By Inter-sentential Attention Mechanism (2022)7.50
• Conversational Speech Recognition By Learning Conversation-level Characteristics (2022)7.50