Multi-scale Octave Convolutions For Robust Speech Recognition

Abstract

We propose a multi-scale octave convolution layer to learn robust speech representations efficiently. Octave convolutions were introduced by Chen et al [1] in the computer vision field to reduce the spatial redundancy of the feature maps by decomposing the output of a convolutional layer into feature maps at two different spatial resolutions, one octave apart. This approach improved the efficiency as well as the accuracy of the CNN models. The accuracy gain was attributed to the enlargement of the receptive field in the original input space. We argue that octave convolutions likewise improve the robustness of learned representations due to the use of average pooling in the lower resolution group, acting as a low-pass filter. We test this hypothesis by evaluating on two noisy speech corpora - Aurora-4 and AMI. We extend the octave convolution concept to multiple resolution groups and multiple octaves. To evaluate the robustness of the inferred representations, we report the similarity b

Stats

• citations8
• S2 citations—
• github stars0
• HF likes0
• heat score7.16
• arxiv keyrownicka2019multi

Related papers

• Very Deep Convolutional Neural Networks For Robust Speech Recognition (2016)12.40
• Multi-time-scale Convolution For Emotion Recognition From Speech Audio Signals (2020)11.67
• Multi-quartznet: Multi-resolution Convolution For Speech Recognition With Multi-layer Feature Fusion (2020)6.34
• An Empirical Analysis Of Speech Self-supervised Learning At Multiple Resolutions (2024)0.00
• Learning Multiscale Features Directly From Waveforms (2016)0.00
• Residual Speaker Representation For One-shot Voice Conversion (2023)0.00
• Robust Raw Waveform Speech Recognition Using Relevance Weighted Representations (2020)3.58
• Mfa-conformer: Multi-scale Feature Aggregation Conformer For Automatic Speaker Verification (2022)15.46