Unsupervised Harmonic Parameter Estimation Using Differentiable DSP And Spectral Optimal Transport

Abstract

In neural audio signal processing, pitch conditioning has been used to enhance the performance of synthesizers. However, jointly training pitch estimators and synthesizers is a challenge when using standard audio-to-audio reconstruction loss, leading to reliance on external pitch trackers. To address this issue, we propose using a spectral loss function inspired by optimal transportation theory that minimizes the displacement of spectral energy. We validate this approach through an unsupervised autoencoding task that fits a harmonic template to harmonic signals. We jointly estimate the fundamental frequency and amplitudes of harmonics using a lightweight encoder and reconstruct the signals using a differentiable harmonic synthesizer. The proposed approach offers a promising direction for improving unsupervised parameter estimation in neural audio applications.

Stats

• citations5
• S2 citations—
• github stars0
• HF likes0
• heat score5.84
• arxiv keytorres2023unsupervised

Related papers

• Differentiable WORLD Synthesizer-based Neural Vocoder With Application To End-to-end Audio Style Transfer (2022)0.00
• Noise-robust Dsp-assisted Neural Pitch Estimation With Very Low Complexity (2023)5.24
• Stabilizing Training With Soft Dynamic Time Warping: A Case Study For Pitch Class Estimation With Weakly Aligned Targets (2023)0.00
• Unsupervised Music Source Separation Using Differentiable Parametric Source Models (2022)10.97
• Cross-domain Neural Pitch And Periodicity Estimation (2023)4.88
• Fast, High-quality And Parameter-efficient Articulatory Synthesis Using Differentiable DSP (2024)2.26
• Towards Improving Harmonic Sensitivity And Prediction Stability For Singing Melody Extraction (2023)0.00
• Self-supervised Disentanglement Of Harmonic And Rhythmic Features In Music Audio Signals (2023)0.00