Multi-modal Mutual Information Maximization: A Novel Approach For Unsupervised Deep Cross-modal Hashing

Abstract

In this paper, we adopt the maximizing mutual information (MI) approach to tackle the problem of unsupervised learning of binary hash codes for efficient cross-modal retrieval. We proposed a novel method, dubbed Cross-Modal Info-Max Hashing (CMIMH). First, to learn informative representations that can preserve both intra- and inter-modal similarities, we leverage the recent advances in estimating variational lower-bound of MI to maximize the MI between the binary representations and input features and between binary representations of different modalities. By jointly maximizing these MIs under the assumption that the binary representations are modelled by multivariate Bernoulli distributions, we can learn binary representations, which can preserve both intra- and inter-modal similarities, effectively in a mini-batch manner with gradient descent. Furthermore, we find out that trying to minimize the modality gap by learning similar binary representations for the same instance from differ

Stats

• citations39
• S2 citations—
• github stars0
• HF likes0
• heat score12.02
• arxiv keyhoang2021multi

Related papers

• Unsupervised Multi-modal Hashing For Cross-modal Retrieval (2019)8.35
• Cross-modal Image Retrieval With Deep Mutual Information Maximization (2021)9.59
• Unsupervised Deep Cross-modality Spectral Hashing (2020)11.39
• Fusion-supervised Deep Cross-modal Hashing (2019)8.60
• Weakly-paired Cross-modal Hashing (2019)0.00
• Discriminative Supervised Hashing For Cross-modal Similarity Search (2018)7.81
• Joint Cluster Unary Loss For Efficient Cross-modal Hashing (2019)5.84
• Deep Cross-modal Hashing Via Margin-dynamic-softmax Loss (2020)0.00