Automated Audio Captioning And Language-based Audio Retrieval

Abstract

This project involved participation in the DCASE 2022 Competition (Task 6) which had two subtasks: (1) Automated Audio Captioning and (2) Language-Based Audio Retrieval. The first subtask involved the generation of a textual description for audio samples, while the goal of the second was to find audio samples within a fixed dataset that match a given description. For both subtasks, the Clotho dataset was used. The models were evaluated on BLEU1, BLEU2, BLEU3, ROUGEL, METEOR, CIDEr, SPICE, and SPIDEr scores for audio captioning and R1, R5, R10 and mARP10 scores for audio retrieval. We have conducted a handful of experiments that modify the baseline models for these tasks. Our final architecture for Automated Audio Captioning is close to the baseline performance, while our model for Language-Based Audio Retrieval has surpassed its counterpart.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keygomes2022automated

Related papers

• Advancing Natural-language Based Audio Retrieval With Passt And Large Audio-caption Data Sets (2023)0.00
• Clotho: An Audio Captioning Dataset (2019)17.70
• The NTT DCASE2020 Challenge Task 6 System: Automated Audio Captioning With Keywords And Sentence Length Estimation (2020)0.00
• Improving Natural-language-based Audio Retrieval With Transfer Learning And Audio & Text Augmentations (2022)0.00
• An Encoder-decoder Based Audio Captioning System With Transfer And Reinforcement Learning (2021)0.00
• Performance Improvement Of Language-queried Audio Source Separation Based On Caption Augmentation From Large Language Models For DCASE Challenge 2024 Task 9 (2024)0.00
• Enhancing Automated Audio Captioning Via Large Language Models With Optimized Audio Encoding (2024)5.24
• RECAP: Retrieval-augmented Audio Captioning (2023)9.41