CanItEdit

Can It Edit? Evaluating the Ability of Large Language Models to Follow Code Editing Instructions CanItEdit is a benchmark for evaluating LLMs on instructional code editing, the task of updating a program given a natural language instruction. The benchmark contains 105 hand-crafted Python programs with before and after

Papers using CanItEdit (3)

Generating High-Quality Datasets for Code Editing via Open-Source Language Models2025Applying the Chinese Wall Reverse Engineering Technique to Large Language Model Code Editing2025EfficientEdit: Accelerating Code Editing via Edit-Oriented Speculative Decoding2025