Learning 3D Semantic Scene Graphs From 3D Indoor Reconstructions

Abstract

Scene understanding has been of high interest in computer vision. It encompasses not only identifying objects in a scene, but also their relationships within the given context. With this goal, a recent line of works tackles 3D semantic segmentation and scene layout prediction. In our work we focus on scene graphs, a data structure that organizes the entities of a scene in a graph, where objects are nodes and their relationships modeled as edges. We leverage inference on scene graphs as a way to carry out 3D scene understanding, mapping objects and their relationships. In particular, we propose a learned method that regresses a scene graph from the point cloud of a scene. Our novel architecture is based on PointNet and Graph Convolutional Networks (GCN). In addition, we introduce 3DSSG, a semi-automatically generated dataset, that contains semantically rich scene graphs of 3D scenes. We show the application of our method in a domain-agnostic retrieval task, where graphs serve as an inte

Stats

• citations194
• S2 citations—
• github stars0
• HF likes0
• heat score17.18
• arxiv keywald2020learning

Related papers

• Open-world 3D Scene Graph Generation For Retrieval-augmented Reasoning (2025)0.00
• SCENIR: Visual Semantic Clarity Through Unsupervised Scene Graph Retrieval (2025)0.00
• Compact Scene Graphs For Layout Composition And Patch Retrieval (2019)8.09
• Image-to-image Retrieval By Learning Similarity Between Scene Graphs (2020)12.02
• Generating Holistic 3D Scene Abstractions For Text-based Image Retrieval (2016)9.03
• Triplet-aware Scene Graph Embeddings (2019)7.81
• Scene Graph Embeddings Using Relative Similarity Supervision (2021)7.50
• Sgdraw: Scene Graph Drawing Interface Using Object-oriented Representation (2022)0.00