TAMAS: Benchmarking Adversarial Risks In Multi-agent LLM Systems

Abstract

Large Language Models (LLMs) have demonstrated strong capabilities as autonomous agents through tool use, planning, and decision-making abilities, leading to their widespread adoption across diverse tasks. As task complexity grows, multi-agent LLM systems are increasingly used to solve problems collaboratively. However, safety and security of these systems remains largely under-explored. Existing benchmarks and datasets predominantly focus on single-agent settings, failing to capture the unique vulnerabilities of multi-agent dynamics and co-ordination. To address this gap, we introduce $\textbf\{T\}$hreats and $\textbf\{A\}$ttacks in $\textbf\{M\}$ulti-$\textbf\{A\}$gent $\textbf\{S\}$ystems ($\textbf\{TAMAS\}$), a benchmark designed to evaluate the robustness and safety of multi-agent LLM systems. TAMAS includes five distinct scenarios comprising 300 adversarial instances across six attack types and 211 tools, along with 100 harmless tasks. We assess system performance acr