Reinforcement Learning-based Market Making As A Stochastic Control On Non-stationary Limit Order Book Dynamics

Abstract

Reinforcement Learning has emerged as a promising framework for developing adaptive and data-driven strategies, enabling market makers to optimize decision-making policies based on interactions with the limit order book environment. This paper explores the integration of a reinforcement learning agent in a market-making context, where the underlying market dynamics have been explicitly modeled to capture observed stylized facts of real markets, including clustered order arrival times, non-stationary spreads and return drifts, stochastic order quantities and price volatility. These mechanisms aim to enhance stability of the resulting control agent, and serve to incorporate domain-specific knowledge into the agent policy learning process. Our contributions include a practical implementation of a market making agent based on the Proximal-Policy Optimization (PPO) algorithm, alongside a comparative evaluation of the agent's performance under varying market conditions via a simulator-based

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzimmer2025reinforcement

Related papers

• Multi-agent Reinforcement Learning For Market Making: Competition Without Collusion (2025)0.00
• Towards Inverse Reinforcement Learning For Limit Order Book Dynamics (2019)0.00
• Robust Market Making Via Adversarial Reinforcement Learning (2020)0.00
• Robust Risk-sensitive Reinforcement Learning Agents For Trading Markets (2021)0.00
• A Simple Learning Agent Interacting With An Agent-based Market Model (2022)6.77
• Learning And Calibrating Heterogeneous Bounded Rational Market Behaviour With Multi-agent Reinforcement Learning (2024)0.00
• Adversarial Recovery Of Agent Rewards From Latent Spaces Of The Limit Order Book (2019)0.00
• Actively Learning Reinforcement Learning: A Stochastic Optimal Control Approach (2023)0.00