Pretrained Conformers For Audio Fingerprinting And Retrieval

Abstract

Conformers have shown great results in speech processing due to their ability to capture both local and global interactions. In this work, we utilize a self-supervised contrastive learning framework to train conformer-based encoders that are capable of generating unique embeddings for small segments of audio, generalizing well to previously unseen data. We achieve state-of-the-art results for audio retrieval tasks while using only 3 seconds of audio to generate embeddings. Our models are almost completely immune to temporal misalignments and achieve state-of-the-art results in cases of other audio distortions such as noise, reverb or extreme temporal stretching. Code and models are made publicly available and the results are easy to reproduce as we train and test using popular and freely available datasets of different sizes.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyaltwlkany2025pretrained

Related papers

• Neural Audio Fingerprint For High-specific Audio Retrieval Based On Contrastive Learning (2020)11.58
• Simultaneously Learning Robust Audio Embeddings And Balanced Hash Codes For Query-by-example (2022)6.34
• Language-based Audio Retrieval With Converging Tied Layers And Contrastive Loss (2022)2.26
• Matching Text And Audio Embeddings: Exploring Transfer-learning Strategies For Language-based Audio Retrieval (2022)0.00
• Robust And Lightweight Audio Fingerprint For Automatic Content Recognition (2023)0.00
• Estimated Audio-caption Correspondences Improve Language-based Audio Retrieval (2024)0.00
• On The Effect Of Data-augmentation On Local Embedding Properties In The Contrastive Learning Of Music Audio Representations (2024)5.24
• Deep Conditional Representation Learning For Drum Sample Retrieval By Vocalisation (2022)0.00