The Solution For The CVPR 2023 1st Foundation Model Challenge-track2

Abstract

In this paper, we propose a solution for cross-modal transportation retrieval. Due to the cross-domain problem of traffic images, we divide the problem into two sub-tasks of pedestrian retrieval and vehicle retrieval through a simple strategy. In pedestrian retrieval tasks, we use IRRA as the base model and specifically design an Attribute Classification to mine the knowledge implied by attribute labels. More importantly, We use the strategy of Inclusion Relation Matching to make the image-text pairs with inclusion relation have similar representation in the feature space. For the vehicle retrieval task, we use BLIP as the base model. Since aligning the color attributes of vehicles is challenging, we introduce attribute-based object detection techniques to add color patch blocks to vehicle images for color data augmentation. This serves as strong prior information, helping the model perform the image-text alignment. At the same time, we incorporate labeled attributes into the image-tex

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyxu2024the

Related papers

• Connecting Language And Vision For Natural Language-based Vehicle Retrieval (2021)15.14
• Self-enhancement Improves Text-image Retrieval In Foundation Visual-language Models (2023)1.56
• Cross-view Image Retrieval -- Ground To Aerial Image Retrieval Through Deep Learning (2020)5.24
• All You Can Embed: Natural Language Based Vehicle Retrieval With Spatio-temporal Transformers (2021)9.19
• Cross-modal Implicit Relation Reasoning And Aligning For Text-to-image Person Retrieval (2023)18.15
• CMIR-NET : A Deep Learning Based Model For Cross-modal Retrieval In Remote Sensing (2019)13.34
• BEV-TSR: Text-scene Retrieval In BEV Space For Autonomous Driving (2024)6.34
• Progressive Feature Mining And External Knowledge-assisted Text-pedestrian Image Retrieval (2023)5.84