CTF

The 'CTF' dataset/benchmark consists of cybersecurity challenges that are used to evaluate the robustness and generalization of agentic large language models (LLMs) through the generation of semantically-equivalent challenge families via program transformations.

Papers using CTF (2)

Capture The Flags: Family-based Evaluation Of Agentic Llms Via Semantics-preserving Transformations2026How Far Have We Gone In Vulnerability Detection Using Large Language Models2023