Visual Lifelog Retrieval Through Captioning-enhanced Interpretation

Abstract

People often struggle to remember specific details of past experiences, which can lead to the need to revisit these memories. Consequently, lifelog retrieval has emerged as a crucial application. Various studies have explored methods to facilitate rapid access to personal lifelogs for memory recall assistance. In this paper, we propose a Captioning-Integrated Visual Lifelog (CIVIL) Retrieval System for extracting specific images from a user's visual lifelog based on textual queries. Unlike traditional embedding-based methods, our system first generates captions for visual lifelogs and then utilizes a text embedding model to project both the captions and user queries into a shared vector space. Visual lifelogs, captured through wearable cameras, provide a first-person viewpoint, necessitating the interpretation of the activities of the individual behind the camera rather than merely describing the scene. To address this, we introduce three distinct approaches: the single caption method,

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyshih2025visual

Related papers

• Lifeir At The NTCIR-18 Lifelog-6 Task (2025)0.00
• Scene Graph Based Image Retrieval -- A Case Study On The CLEVR Dataset (2019)0.00
• Leveraging Large Vision-language Model As User Intent-aware Encoder For Composed Image Retrieval (2024)3.58
• The State-of-the-art In Lifelog Retrieval: A Review Of Progress At The ACM Lifelog Search Challenge Workshop 2022-24 (2025)0.00
• Towards Retrieval-augmented Architectures For Image Captioning (2024)9.41
• Retrieval-augmented Image Captioning (2023)11.29
• Linear Alignment Of Vision-language Models For Image Captioning (2023)0.00
• VQA4CIR: Boosting Composed Image Retrieval With Visual Question Answering (2023)5.24