← all papers · overview

Vincicoder: Unifying Multimodal Code Generation Via Coarse-to-fine Visual Reinforcement Learning

Abstract

Multimodal code generation has garnered significant interest within the research community. Despite the notable success of recent vision-language models (VLMs) on specialized tasks like chart-to-code generation, their reliance on single-task training regimens fosters a narrow paradigm that hinders the development of generalized \textbf\{VI\}sio\textbf\{N\} \textbf\{C\}ode \textbf\{I\}ntelligence. In this work, we introduce \textbf\{VinciCoder\}, a unified multimodal code generation model that addresses this limitation via a two-stage training framework. We begin by constructing a large-scale Supervised Finetuning (SFT) corpus comprising 1.6M image-code pairs for tasks involving direct code generation and visual-based code refinement. Subsequently, we introduce a Visual Reinforcement Learning (ViRL) strategy, which employs a coarse-to-fine reward mechanism to improve visual fidelity by calculating visual similarity across local and global image patches. Extensive experiments on diverse

Related papers

Ranked by semantic similarity — how closely each paper's abstract matches this one (100% = near-identical topic).