A Simple Learning Agent Interacting With An Agent-based Market Model

Abstract

We consider the learning dynamics of a single reinforcement learning optimal execution trading agent when it interacts with an event driven agent-based financial market model. Trading takes place asynchronously through a matching engine in event time. The optimal execution agent is considered at different levels of initial order-sizes and differently sized state spaces. The resulting impact on the agent-based model and market are considered using a calibration approach that explores changes in the empirical stylised facts and price impact curves. Convergence, volume trajectory and action trace plots are used to visualise the learning dynamics. Here the smaller state space agents had the number of states they visited converge much faster than the larger state space agents, and they were able to start learning to trade intuitively using the spread and volume states. We find that the moments of the model are robust to the impact of the learning agents except for the Hurst exponent, which

Stats

• citations7
• S2 citations—
• github stars0
• HF likes0
• heat score6.77
• arxiv keydicks2022a

Related papers

• Learning And Calibrating Heterogeneous Bounded Rational Market Behaviour With Multi-agent Reinforcement Learning (2024)0.00
• Robust Risk-sensitive Reinforcement Learning Agents For Trading Markets (2021)0.00
• Reinforcement Learning-based Market Making As A Stochastic Control On Non-stationary Limit Order Book Dynamics (2025)0.00
• Towards Inverse Reinforcement Learning For Limit Order Book Dynamics (2019)0.00
• Inferring Agent Objectives At Different Scales Of A Complex Adaptive System (2017)0.00
• Modelling Stock-market Investors As Reinforcement Learning Agents [correction] (2016)5.84
• Multi-agent Reinforcement Learning For Market Making: Competition Without Collusion (2025)0.00
• The Evolutionary Dynamics Of Independent Learning Agents In Population Games (2020)0.00