Cybench

Cybench is a dataset that contains a collection of cybersecurity challenges used to evaluate the robustness and generalization of agentic large language models through semantics-preserving program transformations.

Papers using Cybench (4)

Capture The Flags: Family-based Evaluation Of Agentic Llms Via Semantics-preserving Transformations2026Pentest-r1: Towards Autonomous Penetration Testing Reasoning Optimized Via Two-stage Reinforcement Learning2025Cyber-Zero: Training Cybersecurity Agents without Runtime2025Enigma: Interactive Tools Substantially Assist LM Agents In Finding Security Vulnerabilities2024