Sample-specific Debiasing For Better Image-text Models

Abstract

Self-supervised representation learning on image-text data facilitates crucial medical applications, such as image classification, visual grounding, and cross-modal retrieval. One common approach involves contrasting semantically similar (positive) and dissimilar (negative) pairs of data points. Drawing negative samples uniformly from the training data set introduces false negatives, i.e., samples that are treated as dissimilar but belong to the same class. In healthcare data, the underlying class distribution is nonuniform, implying that false negatives occur at a highly variable rate. To improve the quality of learned representations, we develop a novel approach that corrects for false negatives. Our method can be viewed as a variant of debiased contrastive learning that uses estimated sample-specific class probabilities. We provide theoretical analysis of the objective function and demonstrate the proposed approach on both image and paired image-text data sets. Our experiments illus

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywang2023sample

Related papers

• Medclip: Contrastive Learning From Unpaired Medical Images And Text (2022)26.02
• Your Negative May Not Be True Negative: Boosting Image-text Matching With False Negative Elimination (2023)14.32
• Negative Sample Is Negative In Its Own Way: Tailoring Negative Sentences For Image-text Retrieval (2021)3.81
• Boosting Weak Positives For Text Based Person Search (2025)0.00
• Support-set Bottlenecks For Video-text Representation Learning (2020)0.00
• Bima: Towards Biases Mitigation For Text-video Retrieval Via Scene Element Guidance (2025)2.26
• Gistembed: Guided In-sample Selection Of Training Negatives For Text Embedding Fine-tuning (2024)0.00
• Approximate Nearest Neighbor Negative Contrastive Learning For Dense Text Retrieval (2020)0.00