Benchmark Leaderboards
Loading leaderboardsβ¦
Loading leaderboardsβ¦
Track which models hold the current SOTA on the benchmarks the field actually uses. Click a column header to sort; the leading row is highlighted.
CTIBench cyber-threat-intelligence benchmark β average score across its sub-tasks (CyNER, APTNER, CyNews, SecMMLU, CyQuiz, MITRE, CVE, and protocol F1). Evaluates LLM knowledge of the cyber-threat landscape.
Metric: Average Score β higher is better Β· Source β
| # β² | Model / Paper | Average Score | Ξ vs SOTA | Links |
|---|---|---|---|---|
| 1 | GPT-4SOTA | 69.6 | β | search β |
| 2 | GPT-3.5-Turbo | 62.6 | -7.00 | search β |
| 3 | Mistral-7B-v0.1 | 58.1 | -11.50 | search β |
| 4 | Zephyr-7B-beta | 57.7 | -11.90 | search β |
| 5 | Vicuna-13B-v1.5 | 57.3 | -12.30 | search β |
| 6 | Mistral-7B-Instruct-v0.1 | 55 | -14.60 | search β |
| 7 | Llama-2-13B | 54.1 | -15.50 | search β |
| 8 | Vicuna-7B-v1.5 | 53 | -16.60 | search β |
| 9 | Llama-2-7B | 50.6 | -19.00 | search β |
| 10 | Llama-2-13B-Chat | 45 | -24.60 | search β |
| 11 | Llama-2-7B-Chat | 44.6 | -25.00 | search β |
| 12 | Falcon-7B | 39.4 | -30.20 | search β |
| 13 | Falcon-7B-Instruct | 37.5 | -32.10 | search β |