Analyzing Deep Cnn-based Utterance Embeddings For Acoustic Model Adaptation

Abstract

We explore why deep convolutional neural networks (CNNs) with small two-dimensional kernels, primarily used for modeling spatial relations in images, are also effective in speech recognition. We analyze the representations learned by deep CNNs and compare them with deep neural network (DNN) representations and i-vectors, in the context of acoustic model adaptation. To explore whether interpretable information can be decoded from the learned representations we evaluate their ability to discriminate between speakers, acoustic conditions, noise type, and gender using the Aurora-4 dataset. We extract both whole model embeddings (to capture the information learned across the whole network) and layer-specific embeddings which enable understanding of the flow of information across the network. We also use learned representations as the additional input for a time-delay neural network (TDNN) for the Aurora-4 and MGB-3 English datasets. We find that deep CNN embeddings outperform DNN embeddings

Stats

• citations7
• S2 citations—
• github stars0
• HF likes0
• heat score6.77
• arxiv keyrownicka2018analyzing

Related papers

• Embeddings For DNN Speaker Adaptive Training (2019)7.16
• An Improved Deep Neural Network For Modeling Speaker Characteristics At Different Temporal Scales (2020)6.34
• Very Deep Convolutional Neural Networks For Robust Speech Recognition (2016)12.40
• Embedding-based Speaker Adaptive Training Of Deep Neural Networks (2017)9.76
• Analyzing Learned Representations Of A Deep ASR Performance Prediction Model (2018)4.52
• Frame-level Speaker Embeddings For Text-independent Speaker Recognition And Analysis Of End-to-end Model (2018)12.17
• Advances In Very Deep Convolutional Neural Networks For LVCSR (2016)0.00
• Analyzing Hidden Representations In End-to-end Automatic Speech Recognition Systems (2017)0.00