Robust Market Making Via Adversarial Reinforcement Learning

Abstract

We show that adversarial reinforcement learning (ARL) can be used to produce market marking agents that are robust to adversarial and adaptively-chosen market conditions. To apply ARL, we turn the well-studied single-agent model of Avellaneda and Stoikov [2008] into a discrete-time zero-sum game between a market maker and adversary. The adversary acts as a proxy for other market participants that would like to profit at the market maker's expense. We empirically compare two conventional single-agent RL agents with ARL, and show that our ARL approach leads to: 1) the emergence of risk-averse behaviour without constraints or domain-specific penalties; 2) significant improvements in performance across a set of standard metrics, evaluated with or without an adversary in the test environment, and; 3) improved robustness to model uncertainty. We empirically demonstrate that our ARL method consistently converges, and we prove for several special cases that the profiles that we converge to cor

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyspooner2020robust

Related papers

• Multi-agent Reinforcement Learning For Market Making: Competition Without Collusion (2025)0.00
• Robust Risk-sensitive Reinforcement Learning Agents For Trading Markets (2021)0.00
• Robust Adversarial Reinforcement Learning Via Bounded Rationality Curricula (2023)0.00
• Robust Reinforcement Learning On State Observations With Learned Optimal Adversary (2021)0.00
• Learning And Calibrating Heterogeneous Bounded Rational Market Behaviour With Multi-agent Reinforcement Learning (2024)0.00
• Robust Model-based Reinforcement Learning With An Adversarial Auxiliary Model (2024)0.00
• Reinforcement Learning-based Market Making As A Stochastic Control On Non-stationary Limit Order Book Dynamics (2025)0.00
• Robust Deep Reinforcement Learning Through Adversarial Loss (2020)0.00