Multi-agent Reinforcement Learning For Market Making: Competition Without Collusion

Abstract

Algorithmic collusion has emerged as a central question in AI: Will the interaction between different AI agents deployed in markets lead to collusion? More generally, understanding how emergent behavior, be it a cartel or market dominance from more advanced bots, affects the market overall is an important research question. We propose a hierarchical multi-agent reinforcement learning framework to study algorithmic collusion in market making. The framework includes a self-interested market maker (Agent~A), which is trained in an uncertain environment shaped by an adversary, and three bottom-layer competitors: the self-interested Agent~B1 (whose objective is to maximize its own PnL), the competitive Agent~B2 (whose objective is to minimize the PnL of its opponent), and the hybrid Agent~B\(^\star\), which can modulate between the behavior of the other two. To analyze how these agents shape the behavior of each other and affect market outcomes, we propose interaction-level metrics that q

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywang2025multi

Related papers

• Robust Market Making Via Adversarial Reinforcement Learning (2020)0.00
• Reinforcement Learning-based Market Making As A Stochastic Control On Non-stationary Limit Order Book Dynamics (2025)0.00
• Robust Risk-sensitive Reinforcement Learning Agents For Trading Markets (2021)0.00
• Learning And Calibrating Heterogeneous Bounded Rational Market Behaviour With Multi-agent Reinforcement Learning (2024)0.00
• Algorithmic Pricing With Independent Learners And Relative Experience Replay (2021)0.00
• Collaboration Of AI Agents Via Cooperative Multi-agent Deep Reinforcement Learning (2019)0.00
• Adversarial Attacks In Cooperative AI (2021)0.00
• Cooperative And Competitive Biases For Multi-agent Reinforcement Learning (2021)2.26