Content-context Factorized Representations For Automated Speech Recognition

Abstract

Deep neural networks have largely demonstrated their ability to perform automated speech recognition (ASR) by extracting meaningful features from input audio frames. Such features, however, may consist not only of information about the spoken language content, but also may contain information about unnecessary contexts such as background noise and sounds or speaker identity, accent, or protected attributes. Such information can directly harm generalization performance, by introducing spurious correlations between the spoken words and the context in which such words were spoken. In this work, we introduce an unsupervised, encoder-agnostic method for factoring speech-encoder representations into explicit content-encoding representations and spurious context-encoding representations. By doing so, we demonstrate improved performance on standard ASR benchmarks, as well as improved performance in both real-world and artificially noisy ASR scenarios.

Stats

• citations6
• S2 citations—
• github stars0
• HF likes0
• heat score6.34
• arxiv keychan2022content

Related papers

• Deep Factorization For Speech Signal (2018)8.82
• Towards Effective And Compact Contextual Representation For Conformer Transducer Speech Recognition Systems (2023)7.16
• Leveraging Acoustic Contextual Representation By Audio-textual Cross-modal Learning For Conversational ASR (2022)0.00
• Mixture Factorized Auto-encoder For Unsupervised Hierarchical Deep Factorization Of Speech Signal (2019)0.00
• Adversarially Learning Disentangled Speech Representations For Robust Multi-factor Voice Conversion (2021)9.92
• Contextual Joint Factor Acoustic Embeddings (2019)2.26
• Contextualized Spoken Word Representations From Convolutional Autoencoders (2020)0.00
• Deep Contextualized Acoustic Representations For Semi-supervised Speech Recognition (2019)14.62