An Empirical Study Of Weakly Supervised Audio Tagging Embeddings For General Audio Representations

Abstract

We study the usability of pre-trained weakly supervised audio tagging (AT) models as feature extractors for general audio representations. We mainly analyze the feasibility of transferring those embeddings to other tasks within the speech and sound domains. Specifically, we benchmark weakly supervised pre-trained models (MobileNetV2 and EfficientNet-B0) against modern self-supervised learning methods (BYOL-A) as feature extractors. Fourteen downstream tasks are used for evaluation ranging from music instrument classification to language classification. Our results indicate that AT pre-trained models are an excellent transfer learning choice for music, event, and emotion recognition tasks. Further, finetuning AT models can also benefit speech-related tasks such as keyword spotting and intent classification.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keydinkel2022an

Related papers

• Supervised And Unsupervised Learning Of Audio Representations For Music Understanding (2022)0.00
• Learning Music Audio Representations Via Weak Language Supervision (2021)10.07
• Integrated Parameter-efficient Tuning For General-purpose Audio Models (2022)0.00
• BYOL For Audio: Self-supervised Learning For General-purpose Audio Representation (2021)15.22
• On The Transferability Of Large-scale Self-supervision To Few-shot Audio Classification (2024)3.58
• Learning Audio And Image Representations With Bio-inspired Trainable Feature Extractors (2018)2.26
• Efficient Large-scale Audio Tagging Via Transformer-to-cnn Knowledge Distillation (2022)17.68
• Attention And Localization Based On A Deep Convolutional Recurrent Model For Weakly Supervised Audio Tagging (2017)11.39