Probabilistic Compositional Embeddings For Multimodal Image Retrieval

Abstract

Existing works in image retrieval often consider retrieving images with one or two query inputs, which do not generalize to multiple queries. In this work, we investigate a more challenging scenario for composing multiple multimodal queries in image retrieval. Given an arbitrary number of query images and (or) texts, our goal is to retrieve target images containing the semantic concepts specified in multiple multimodal queries. To learn an informative embedding that can flexibly encode the semantics of various queries, we propose a novel multimodal probabilistic composer (MPC). Specifically, we model input images and texts as probabilistic embeddings, which can be further composed by a probabilistic composition rule to facilitate image retrieval with multiple multimodal queries. We propose a new benchmark based on the MS-COCO dataset and evaluate our model on various setups that compose multiple images and (or) text queries for multimodal image retrieval. Without bells and whistles, we

Stats

• citations28
• S2 citations—
• github stars25
• HF likes0
• heat score13.80
• arxiv keyneculai2022probabilistic

Related papers

• Probabilistic Embeddings For Cross-modal Retrieval (2021)21.70
• Candidate Set Re-ranking For Composed Image Retrieval With Dual Multi-modal Encoder (2023)2.64
• Composed Multi-modal Retrieval: A Survey Of Approaches And Applications (2025)3.88
• Composing Text And Image For Image Retrieval - An Empirical Odyssey (2018)18.71
• Compositional Learning Of Image-text Query For Image Retrieval (2020)17.87
• Embedding Arithmetic Of Multimodal Queries For Image Retrieval (2021)9.03
• Uncertainty-based Cross-modal Retrieval With Probabilistic Representations (2022)0.00
• Revisiting Cross Modal Retrieval (2018)0.00