ICPC

The 'ICPC' dataset/benchmark contains problems from premier programming competitions and is used to evaluate the reasoning and coding capabilities of Large Language Models (LLMs).

Papers using ICPC (6)

Can Multi-turn Self-refined Single Agent LMs with Retrieval Solve Hard Coding Problems?2025 · 2 citesAetherCode: Evaluating LLMs' Ability to Win In Premier Programming Competitions2025OJBench: A Competition Level Code Benchmark For Large Language Models2025Evaluating and Improving Large Language Models for Competitive Program Generation2025LiveCodeBench Pro: How Do Olympiad Medalists Judge LLMs in Competitive Programming?2025AetherCode: Evaluating LLMs' Ability to Win In Premier Programming Competitions2025