Context-aware Embeddings For Automatic Art Analysis

Abstract

Automatic art analysis aims to classify and retrieve artistic representations from a collection of images by using computer vision and machine learning techniques. In this work, we propose to enhance visual representations from neural networks with contextual artistic information. Whereas visual representations are able to capture information about the content and the style of an artwork, our proposed context-aware embeddings additionally encode relationships between different artistic attributes, such as author, school, or historical period. We design two different approaches for using context in automatic art analysis. In the first one, contextual data is obtained through a multi-task learning model, in which several attributes are trained together to find visual relationships between elements. In the second approach, context is obtained through an art-specific knowledge graph, which encodes relationships between artistic attributes. An exhaustive evaluation of both of our models in

Stats

• citations46
• S2 citations—
• github stars0
• HF likes0
• heat score12.54
• arxiv keygarcia2019context

Related papers

• How To Read Paintings: Semantic Art Understanding With Multi-modal Retrieval (2018)13.93
• Artseek: Deep Artwork Understanding Via Multimodal In-context Reasoning And Late Interaction Retrieval (2025)2.16
• Leveraging Computer Vision Application In Visual Arts: A Case Study On The Use Of Residual Neural Network To Classify And Analyze Baroque Paintings (2022)3.58
• Context Sensitivity Improves Human-machine Visual Alignment (2026)0.00
• Visual Link Retrieval And Knowledge Discovery In Painting Datasets (2020)12.25
• Interpreting Context Of Images Using Scene Graphs (2019)0.00
• Context Embedding Networks (2017)4.52
• Contextclip: Contextual Alignment Of Image-text Pairs On CLIP Visual Representations (2022)5.84