BEV-TSR: Text-scene Retrieval In BEV Space For Autonomous Driving

Abstract

The rapid development of the autonomous driving industry has led to a significant accumulation of autonomous driving data. Consequently, there comes a growing demand for retrieving data to provide specialized optimization. However, directly applying previous image retrieval methods faces several challenges, such as the lack of global feature representation and inadequate text retrieval ability for complex driving scenes. To address these issues, firstly, we propose the BEV-TSR framework which leverages descriptive text as an input to retrieve corresponding scenes in the Bird's Eye View (BEV) space. Then to facilitate complex scene retrieval with extensive text descriptions, we employ a large language model (LLM) to extract the semantic features of the text inputs and incorporate knowledge graph embeddings to enhance the semantic richness of the language embedding. To achieve feature alignment between the BEV feature and language embedding, we propose Shared Cross-modal Embedding with a

Stats

• citations6
• S2 citations—
• github stars0
• HF likes0
• heat score6.34
• arxiv keytang2024bev

Related papers

• C-BEV: Contrastive Bird's Eye View Training For Cross-view Image Retrieval And 3-dof Pose Estimation (2023)0.00
• All You Can Embed: Natural Language Based Vehicle Retrieval With Spatio-temporal Transformers (2021)9.19
• Direction-oriented Visual-semantic Embedding Model For Remote Sensing Image-text Retrieval (2023)11.29
• Beyond Visual Semantics: Exploring The Role Of Scene Text In Image Understanding (2019)9.59
• Connecting Language And Vision For Natural Language-based Vehicle Retrieval (2021)15.14
• Stacmr: Scene-text Aware Cross-modal Retrieval (2020)10.48
• Fast And Efficient Scene Categorization For Autonomous Driving Using Vaes (2022)2.26
• The Solution For The CVPR 2023 1st Foundation Model Challenge-track2 (2024)0.00