Adversarial Hubness In Multi-modal Retrieval

Abstract

Hubness is a phenomenon in high-dimensional vector spaces where a point from the natural distribution is unusually close to many other points. This is a well-known problem in information retrieval that causes some items to accidentally (and incorrectly) appear relevant to many queries. In this paper, we investigate how attackers can exploit hubness to turn any image or audio input in a multi-modal retrieval system into an adversarial hub. Adversarial hubs can be used to inject universal adversarial content (e.g., spam) that will be retrieved in response to thousands of different queries, and also for targeted attacks on queries related to specific, attacker-chosen concepts. We present a method for creating adversarial hubs and evaluate the resulting hubs on benchmark multi-modal retrieval datasets and an image-to-image retrieval system implemented by Pinecone, a popular vector database. For example, in text-caption-to-image retrieval, a single adversarial hub, generated using 100 r

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhang2024adversarial

Related papers

• Neighborretr: Balancing Hub Centrality In Cross-modal Retrieval (2025)4.17
• Hubness Reduction With Dual Bank Sinkhorn Normalization For Cross-modal Retrieval (2025)0.95
• Balance Act: Mitigating Hubness In Cross-modal Retrieval With Query And Gallery Banks (2023)8.46
• HAL: Improved Text-image Matching By Mitigating Visual Semantic Hubs (2019)10.48
• Unsupervised Multi-criteria Adversarial Detection In Deep Image Retrieval (2023)0.00
• Hashgan:attention-aware Deep Adversarial Hashing For Cross Modal Retrieval (2017)15.34
• Hv-attack: Hierarchical Visual Attack For Multimodal Retrieval Augmented Generation (2025)0.00
• Integrating Information Theory And Adversarial Learning For Cross-modal Retrieval (2021)10.97