Gaca-dit: Diffusion-based Dance-to-music Generation With Genre-adaptive Rhythm And Context-aware Alignment

Abstract

Dance-to-music (D2M) generation aims to automatically compose music that is rhythmically and temporally aligned with dance movements. Existing methods typically rely on coarse rhythm embeddings, such as global motion features or binarized joint-based rhythm values, which discard fine-grained motion cues and result in weak rhythmic alignment. Moreover, temporal mismatches introduced by feature downsampling further hinder precise synchronization between dance and music. To address these problems, we propose \textbf\{GACA-DiT\}, a diffusion transformer-based framework with two novel modules for rhythmically consistent and temporally aligned music generation. First, a \textbf\{genre-adaptive rhythm extraction\} module combines multi-scale temporal wavelet analysis and spatial phase histograms with adaptive joint weighting to capture fine-grained, genre-specific rhythm patterns. Second, a \textbf\{context-aware temporal alignment\} module resolves temporal mismatches using learnable context

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywang2025gaca

Related papers

• Motionrag-diff: A Retrieval-augmented Diffusion Framework For Long-term Music-to-dance Generation (2025)0.00
• Diffrhythm+: Controllable And Flexible Full-length Song Generation With Preference Optimization (2025)3.58
• Musicldm: Enhancing Novelty In Text-to-music Generation Using Beat-synchronous Mixup Strategies (2023)13.55
• Diffrhythm 2: Efficient And High Fidelity Song Generation Via Block Flow Matching (2025)0.00
• Diffrhythm: Blazingly Fast And Embarrassingly Simple End-to-end Full-length Song Generation With Latent Diffusion (2025)0.00
• Gesture2music: A Low-latency Real-time Framework For Continuous Gesture-driven Music Generation (2026)0.00
• Diff-a-riff: Musical Accompaniment Co-creation Via Latent Diffusion Models (2024)0.00
• Samuel: Efficient Vocal-conditioned Music Generation Via Soft Alignment Attention And Latent Diffusion (2025)0.00