Multiple F0 Estimation In Vocal Ensembles Using Convolutional Neural Networks

Abstract

This paper addresses the extraction of multiple F0 values from polyphonic and a cappella vocal performances using convolutional neural networks (CNNs). We address the major challenges of ensemble singing, i.e., all melodic sources are vocals and singers sing in harmony. We build upon an existing architecture to produce a pitch salience function of the input signal, where the harmonic constant-Q transform (HCQT) and its associated phase differentials are used as an input representation. The pitch salience function is subsequently thresholded to obtain a multiple F0 estimation output. For training, we build a dataset that comprises several multi-track datasets of vocal quartets with F0 annotations. This work proposes and evaluates a set of CNNs for this task in diverse scenarios and data configurations, including recordings with additional reverb. Our models outperform a state-of-the-art method intended for the same music genre when evaluated with an increased F0 resolution, as well as a

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keycuesta2020multiple

Related papers

• Real-time Pitch/f0 Detection Using Spectrogram Images And Convolutional Neural Networks (2025)0.00
• Singing Voice Separation And Vocal F0 Estimation Based On Mutual Combination Of Robust Principal Component Analysis And Subharmonic Summation (2016)10.74
• Vocal Melody Extraction Using Patch-based CNN (2018)12.47
• Fast And High-quality Singing Voice Synthesis System Based On Convolutional Neural Networks (2019)8.82
• Multi-band Multi-resolution Fully Convolutional Neural Networks For Singing Voice Separation (2019)5.84
• DEEPF0: End-to-end Fundamental Frequency Estimation For Music And Speech Signals (2021)10.35
• Towards End-to-end F0 Voice Conversion Based On Dual-gan With Convolutional Wavelet Kernels (2021)5.84
• Human Voice Pitch Estimation: A Convolutional Network With Auto-labeled And Synthetic Data (2023)0.00