Enhancing Unsupervised Audio Representation Learning Via Adversarial Sample Generation

Abstract

Existing audio analysis methods generally first transform the audio stream to spectrogram, and then feed it into CNN for further analysis. A standard CNN recognizes specific visual patterns over feature map, then pools for high-level representation, which overlooks the positional information of recognized patterns. However, unlike natural image, the semantic of an audio spectrogram is sensitive to positional change, as its vertical and horizontal axes indicate the frequency and temporal information of the audio, instead of naive rectangular coordinates. Thus, the insensitivity of CNN to positional change plays a negative role on audio spectrogram encoding. To address this issue, this paper proposes a new self-supervised learning mechanism, which enhances the audio representation by first generating adversarial samples (\textit\{i.e.\}, negative samples), then driving CNN to distinguish the embeddings of negative pairs in the latent space. Extensive experiments show that the proposed ap

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keypan2023enhancing

Related papers

• Targeted Adversarial Examples For Black Box Audio Systems (2018)15.75
• High-fidelity Audio Generation And Representation Learning With Guided Adversarial Autoencoder (2020)8.35
• Audioldm 2: Learning Holistic Audio Generation With Self-supervised Pretraining (2023)0.00
• Learning Speech Representations From Raw Audio By Joint Audiovisual Self-supervision (2020)0.00
• BYOL For Audio: Self-supervised Learning For General-purpose Audio Representation (2021)15.22
• Bandwidth Extension On Raw Audio Via Generative Adversarial Networks (2019)0.00
• Learning Robust Heterogeneous Signal Features From Parallel Neural Network For Audio Sentiment Analysis (2018)0.00
• Diverse Audio Captioning Via Adversarial Training (2021)0.00