Diverse Audio Embeddings -- Bringing Features Back Outperforms CLAP!

Abstract

With the advent of modern AI architectures, a shift has happened towards end-to-end architectures. This pivot has led to neural architectures being trained without domain-specific biases/knowledge, optimized according to the task. We in this paper, learn audio embeddings via diverse feature representations, in this case, domain-specific. For the case of audio classification over hundreds of categories of sound, we learn robust separate embeddings for diverse audio properties such as pitch, timbre, and neural representation, along with also learning it via an end-to-end architecture. We observe handcrafted embeddings, e.g., pitch and timbre-based, although on their own, are not able to beat a fully end-to-end representation, yet adding these together with end-to-end embedding helps us, significantly improve performance. This work would pave the way to bring some domain expertise with end-to-end models to learn robust, diverse representations, surpassing the performance of just training

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyverma2023diverse

Related papers

• Transformation Of Audio Embeddings Into Interpretable, Concept-based Representations (2025)2.26
• M2D-CLAP: Masked Modeling Duo Meets CLAP For Learning General-purpose Audio-language Representation (2024)7.81
• Brewclip: A Bifurcated Representation Learning Framework For Audio-visual Retrieval (2024)0.00
• Towards Evaluating Generative Audio: Insights From Neural Audio Codec Embedding Distances (2025)0.00
• PERSA+: A Deep Learning Front-end For Context-agnostic Audio Classification (2021)0.00
• Enclap: Combining Neural Audio Codec And Audio-text Joint Embedding For Automated Audio Captioning (2024)14.03
• Conditional End-to-end Audio Transforms (2018)8.82
• Investigating The Sensitivity Of Pre-trained Audio Embeddings To Common Effects (2025)4.52