← all papers · overview

Gtr-cot: Graph Traversal As Visual Chain Of Thought For Molecular Structure Recognition

Abstract

Optical Chemical Structure Recognition (OCSR) is essential for converting molecular images into machine-readable formats. While recent vision-language models (VLMs) have shown promise, their image-captioning approach often struggles with complex molecular structures and inconsistent annotations. To address these issues, we introduce GTR-VL, featuring two key innovations: (1) the \textit\{Graph Traversal as Visual Chain of Thought\} mechanism that emulates human reasoning by incrementally parsing molecular graphs through sequential atom-bond predictions, and (2) the data-centric \textit\{Faithfully Recognize What You've Seen\} principle, which aligns abbreviated structures in images with their expanded annotations. For hand-drawn OCSR tasks, where datasets lack graph annotations and only provide final SMILES, we apply reinforcement learning using the GRPO method, introducing reward mechanisms like format reward, graph reward, and SMILES reward. This approach significantly enhances perfo

Related papers

Ranked by semantic similarity — how closely each paper's abstract matches this one (100% = near-identical topic).