Scene Graph Embeddings Using Relative Similarity Supervision

Abstract

Scene graphs are a powerful structured representation of the underlying content of images, and embeddings derived from them have been shown to be useful in multiple downstream tasks. In this work, we employ a graph convolutional network to exploit structure in scene graphs and produce image embeddings useful for semantic image retrieval. Different from classification-centric supervision traditionally available for learning image representations, we address the task of learning from relative similarity labels in a ranking context. Rooted within the contrastive learning paradigm, we propose a novel loss function that operates on pairs of similar and dissimilar images and imposes relative ordering between them in embedding space. We demonstrate that this Ranking loss, coupled with an intuitive triple sampling strategy, leads to robust representations that outperform well-known contrastive losses on the retrieval task. In addition, we provide qualitative evidence of how retrieved results t

Stats

• citations9
• S2 citations—
• github stars0
• HF likes0
• heat score7.50
• arxiv keymaheshwari2021scene

Related papers

• Triplet-aware Scene Graph Embeddings (2019)7.81
• Image-to-image Retrieval By Learning Similarity Between Scene Graphs (2020)12.02
• SCENIR: Visual Semantic Clarity Through Unsupervised Scene Graph Retrieval (2025)0.00
• Compact Scene Graphs For Layout Composition And Patch Retrieval (2019)8.09
• SPAN: Learning Similarity Between Scene Graphs And Images With Transformers (2023)0.00
• Learning 3D Semantic Scene Graphs From 3D Indoor Reconstructions (2020)17.18
• Beyond Supervised Vs. Unsupervised: Representative Benchmarking And Analysis Of Image Representation Learning (2022)8.35
• VERSE: Versatile Graph Embeddings From Similarity Measures (2018)17.42