Variational Information Bottleneck For Effective Low-resource Audio Classification

Abstract

Large-scale deep neural networks (DNNs) such as convolutional neural networks (CNNs) have achieved impressive performance in audio classification for their powerful capacity and strong generalization ability. However, when training a DNN model on low-resource tasks, it is usually prone to overfitting the small data and learning too much redundant information. To address this issue, we propose to use variational information bottleneck (VIB) to mitigate overfitting and suppress irrelevant information. In this work, we conduct experiments ona 4-layer CNN. However, the VIB framework is ready-to-use and could be easily utilized with many other state-of-the-art network architectures. Evaluation on a few audio datasets shows that our approach significantly outperforms baseline methods, yielding more than 5.0% improvement in terms of classification accuracy in some low-source settings.

Stats

• citations10
• S2 citations—
• github stars0
• HF likes0
• heat score7.81
• arxiv keysi2021variational

Related papers

• Anti-spoofing Using Transfer Learning With Variational Information Bottleneck (2022)9.41
• Reducing Model Complexity For DNN Based Large-scale Audio Classification (2017)9.59
• A Deep Neural Network For Audio Classification With A Classifier Attention Mechanism (2020)0.00
• Audio Classification Of Low Feature Spectrograms Utilizing Convolutional Neural Networks (2024)5.84
• Deep Learning For Audio Transcription On Low-resource Datasets (2018)0.00
• Towards Speech Enhancement Using A Variational U-net Architecture (2020)7.81
• Audio-based Music Classification With Densenet And Data Augmentation (2019)10.48
• On The Performance Of Residual Block Design Alternatives In Convolutional Neural Networks For End-to-end Audio Classification (2019)0.00