Exploring Efficient-tuned Learning Audio Representation Method From Brivl

Abstract

Recently, researchers have gradually realized that in some cases, the self-supervised pre-training on large-scale Internet data is better than that of high-quality/manually labeled data sets, and multimodal/large models are better than single or bimodal/small models. In this paper, we propose a robust audio representation learning method WavBriVL based on Bridging-Vision-and-Language (BriVL). WavBriVL projects audio, image and text into a shared embedded space, so that multi-modal applications can be realized. We demonstrate the qualitative evaluation of the image generated from WavBriVL as a shared embedded space, with the main purposes of this paper:(1) Learning the correlation between audio and image;(2) Explore a new way of image generation, that is, use audio to generate pictures. Experimental results show that this method can effectively generate appropriate images from audio.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyfang2023exploring

Related papers

• Unibrivl: Robust Universal Representation And Generation Of Audio Driven Diffusion Models (2023)2.26
• I Hear Your True Colors: Image Guided Audio Generation (2022)0.00
• WAVE: Learning Unified & Versatile Audio-visual Embeddings With Multimodal LLM (2025)0.00
• BYOL For Audio: Self-supervised Learning For General-purpose Audio Representation (2021)15.22
• Avlnet: Learning Audio-visual Language Representations From Instructional Videos (2020)12.87
• Learning Speech Representations From Raw Audio By Joint Audiovisual Self-supervision (2020)0.00
• Audio-enhanced Vision-language Modeling With Latent Space Broadening For High Quality Data Expansion (2025)0.00
• Brewclip: A Bifurcated Representation Learning Framework For Audio-visual Retrieval (2024)0.00