Deterministic Limit Of Temporal Difference Reinforcement Learning For Stochastic Games

Abstract

Reinforcement learning in multiagent systems has been studied in the fields of economic game theory, artificial intelligence and statistical physics by developing an analytical understanding of the learning dynamics (often in relation to the replicator dynamics of evolutionary game theory). However, the majority of these analytical studies focuses on repeated normal form games, which only have a single environmental state. Environmental dynamics, i.e., changes in the state of an environment affecting the agents' payoffs has received less attention, lacking a universal method to obtain deterministic equations from established multistate reinforcement learning algorithms. In this work we present a novel methodological extension, separating the interaction from the adaptation time scale, to derive the deterministic limit of a general class of reinforcement learning algorithms, called temporal difference learning. This form of learning is equipped to function in more realistic multistate

Stats

• citations52
• S2 citations—
• github stars0
• HF likes0
• heat score12.93
• arxiv keybarfuss2018deterministic

Related papers

• Independent Learning In Stochastic Games (2021)6.77
• Loss Dynamics Of Temporal Difference Reinforcement Learning (2023)0.00
• The Evolutionary Dynamics Of Independent Learning Agents In Population Games (2020)0.00
• Finite-time Performance Of Distributed Temporal Difference Learning With Linear Function Approximation (2019)9.59
• A Black-box Approach For Non-stationary Multi-agent Reinforcement Learning (2023)0.00
• Decentralized Multi-agent Reinforcement Learning For Continuous-space Stochastic Games (2023)5.24
• Markov Decision Processes Under External Temporal Processes (2023)0.00
• Control Theoretic Analysis Of Temporal Difference Learning (2021)0.00