Unibrivl: Robust Universal Representation And Generation Of Audio Driven Diffusion Models

Abstract

Multimodal large models have been recognized for their advantages in various performance and downstream tasks. The development of these models is crucial towards achieving general artificial intelligence in the future. In this paper, we propose a novel universal language representation learning method called UniBriVL, which is based on Bridging-Vision-and-Language (BriVL). Universal BriVL embeds audio, image, and text into a shared space, enabling the realization of various multimodal applications. Our approach addresses major challenges in robust language (both text and audio) representation learning and effectively captures the correlation between audio and image. Additionally, we demonstrate the qualitative evaluation of the generated images from UniBriVL, which serves to highlight the potential of our approach in creating images from audio. Overall, our experimental results demonstrate the efficacy of UniBriVL in downstream tasks and its ability to choose appropriate images from au

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keyfang2023unibrivl

Related papers

• Exploring Efficient-tuned Learning Audio Representation Method From Brivl (2023)0.00
• Uniaudio: An Audio Foundation Model Toward Universal Audio Generation (2023)5.56
• Unispeaker: A Unified Approach For Multimodality-driven Speaker Generation (2025)2.26
• Multi-modality Associative Bridging Through Memory: Speech Sound Recollected From Face Video (2022)12.10
• Uniwav: Towards Unified Pre-training For Speech Representation Learning And Generation (2025)0.00
• M\(^{2}\)ugen: Multi-modal Music Understanding And Generation With The Power Of Large Language Models (2023)0.00
• Audio-enhanced Vision-language Modeling With Latent Space Broadening For High Quality Data Expansion (2025)0.00
• Multimodal Audio-textual Architecture For Robust Spoken Language Understanding (2023)0.00