Adversarial Representation Learning For Text-to-image Matching

Abstract

For many computer vision applications such as image captioning, visual question answering, and person search, learning discriminative feature representations at both image and text level is an essential yet challenging problem. Its challenges originate from the large word variance in the text domain as well as the difficulty of accurately measuring the distance between the features of the two modalities. Most prior work focuses on the latter challenge, by introducing loss functions that help the network learn better feature representations but fail to account for the complexity of the textual input. With that in mind, we introduce TIMAM: a Text-Image Modality Adversarial Matching approach that learns modality-invariant feature representations using adversarial and cross-modal matching objectives. In addition, we demonstrate that BERT, a publicly-available language model that extracts word embeddings, can successfully be applied in the text-to-image matching domain. The proposed approac

Stats

• citations203
• S2 citations—
• github stars0
• HF likes0
• heat score17.32
• arxiv keysarafianos2019adversarial

Related papers

• Bringing Multimodality To Amazon Visual Search System (2024)6.34
• Enhancing Image-text Matching With Adaptive Feature Aggregation (2024)6.34
• ALADIN: Distilling Fine-grained Alignment Scores For Efficient Image-text Matching And Retrieval (2022)14.00
• Integrating Information Theory And Adversarial Learning For Cross-modal Retrieval (2021)10.97
• Transformer Reasoning Network For Image-text Matching And Retrieval (2020)16.15
• MVAM: Multi-view Attention Method For Fine-grained Image-text Matching (2024)0.00
• Fashionbert: Text And Image Matching With Adaptive Loss For Cross-modal Retrieval (2020)15.16
• Cross-modal Full-mode Fine-grained Alignment For Text-to-image Person Retrieval (2025)2.23