Speaker Conditioning Of Acoustic Models Using Affine Transformation For Multi-speaker Speech Recognition

Abstract

This study addresses the problem of single-channel Automatic Speech Recognition of a target speaker within an overlap speech scenario. In the proposed method, the hidden representations in the acoustic model are modulated by speaker auxiliary information to recognize only the desired speaker. Affine transformation layers are inserted into the acoustic model network to integrate speaker information with the acoustic features. The speaker conditioning process allows the acoustic model to perform computation in the context of target-speaker auxiliary information. The proposed speaker conditioning method is a general approach and can be applied to any acoustic model architecture. Here, we employ speaker conditioning on a ResNet acoustic model. Experiments on the WSJ corpus show that the proposed speaker conditioning method is an effective solution to fuse speaker auxiliary information with acoustic features for multi-speaker speech recognition, achieving +9% and +20% relative WER reduction

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyyousefi2021speaker

Related papers

• Time-domain Speech Extraction With Spatial Information And Multi Speaker Conditioning Mechanism (2021)7.81
• Speaker Conditioned Acoustic Modeling For Multi-speaker Conversational ASR (2021)4.52
• End-to-end Multi-speaker Speech Recognition Using Speaker Embeddings And Transfer Learning (2019)9.41
• Distortionless Multi-channel Target Speech Enhancement For Overlapped Speech Recognition (2020)0.00
• Multi-channel Target Speech Extraction With Channel Decorrelation And Target Speaker Adaptation (2020)0.00
• Auxiliary Interference Speaker Loss For Target-speaker Speech Recognition (2019)9.76
• Multi-speaker ASR Combining Non-autoregressive Conformer CTC And Conditional Speaker Chain (2021)11.31
• Streaming Multi-speaker ASR With RNN-T (2020)10.07