CyberMetric (10K) cybermetric Leaderboard
CyberMetric LLM cybersecurity-knowledge benchmark — accuracy on the largest (10,000-question) split, RAG-generated from NIST standards, research papers and security textbooks. Sourced from the FrontierAI Cybersecurity leaderboard's aggregation of published model results. · Metric: Accuracy (higher is better)
| # | Model | Accuracy | Paper |
|---|---|---|---|
| 1 | GPT-4o | 88.89 | — |
| 2 | GPT-4-Turbo | 88.50 | — |
| 3 | Gemini Pro 1.0 | 87.50 | — |
| 4 | Falcon-180B-Chat | 87.00 | — |
| 5 | Mixtral-8x7B-Instruct | 87.00 | — |
| 6 | GPT-3.5-Turbo | 80.30 | — |
| 7 | Mistral-7B-Instruct-v0.2 | 74.82 | — |
| 8 | Gemma-1.1-7B | 73.32 | — |
| 9 | Llama-3-8B-Instruct | 71.25 | — |
| 10 | Flan-T5-XXL | 67.50 | — |
| 11 | Llama 2-70B | 66.10 | — |
| 12 | Zephyr-7B-beta | 65.00 | — |
| 13 | Qwen1.5-MoE-A2.7B | 60.73 | — |
| 14 | Qwen1.5-7B | 59.79 | — |
| 15 | Qwen-7B | 54.09 | — |
| 16 | Phi-2 | 52.13 | — |
| 17 | DeciLM-7B | 50.75 | — |
| 18 | Llama3-ChatQA-1.5-8B | 49.64 | — |
| 19 | Qwen1.5-4B | 40.29 | — |
| 20 | Genstruct-7B | 36.93 | — |
| 21 | Llama-3-8B | 36.00 | — |
| 22 | Gemma-7B | 34.28 | — |
| 23 | Dolly V2 12b BF16 | 27.00 | — |
| 24 | Gemma-2B | 19.18 | — |
| 25 | Phi-3-mini-4k-Instruct | 4.80 | — |