MATS: An Audio Language Model Under Text-only Supervision

Abstract

Large audio-language models (LALMs), built upon powerful Large Language Models (LLMs), have exhibited remarkable audio comprehension and reasoning capabilities. However, the training of LALMs demands a large corpus of audio-language pairs, which requires substantial costs in both data collection and training resources. In this paper, we propose \textbf\{MATS\}, an audio-language multimodal LLM designed to handle \textbf\{M\}ultiple \textbf\{A\}udio task using solely \textbf\{T\}ext-only \textbf\{S\}upervision. By leveraging pre-trained audio-language alignment models such as CLAP, we develop a text-only training strategy that projects the shared audio-language latent space into LLM latent space, endowing the LLM with audio comprehension capabilities without relying on audio data during training. To further bridge the modality gap between audio and language embeddings within CLAP, we propose the \textbf\{S\}trongly-rel\textbf\{a\}ted \textbf\{n\}oisy \textbf\{t\}ext with \textbf\{a\}udi

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywang2025mats

Related papers

• Measuring Audio's Impact On Correctness: Audio-contribution-aware Post-training Of Large Audio Language Models (2025)0.00
• From Alignment To Advancement: Bootstrapping Audio-language Alignment With Synthetic Data (2025)2.26
• Audiotoolagent: An Agentic Framework For Audio-language Models (2025)2.60
• Uniaudio 1.5: Large Language Model-driven Audio Codec Is A Few-shot Audio Task Learner (2024)0.00
• Audiosetcaps: An Enriched Audio-caption Dataset Using Automated Generation Pipeline With Large Audio And Language Models (2024)13.44
• C3LLM: Conditional Multimodal Content Generation Using Large Language Models (2024)0.00
• Large Language Models Are Strong Audio-visual Speech Recognition Learners (2024)9.59
• Do Audio-language Models Understand Linguistic Variations? (2024)0.00