Agent Environment Cycle Games

Abstract

Partially Observable Stochastic Games (POSGs) are the most general and common model of games used in Multi-Agent Reinforcement Learning (MARL). We argue that the POSG model is conceptually ill suited to software MARL environments, and offer case studies from the literature where this mismatch has led to severely unexpected behavior. In response to this, we introduce the Agent Environment Cycle Games (AEC Games) model, which is more representative of software implementation. We then prove it's as an equivalent model to POSGs. The AEC games model is also uniquely useful in that it can elegantly represent both all forms of MARL environments, whereas for example POSGs cannot elegantly represent strictly turn based games like chess.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyterry2020agent

Related papers

• A2C Is A Special Case Of PPO (2022)0.00
• Sample-efficient Reinforcement Learning Of Partially Observable Markov Games (2022)0.00
• On Convex Optimal Value Functions For Posgs (2023)0.00
• Stackelberg Games For Learning Emergent Behaviors During Competitive Autocurricula (2023)5.84
• The Surprising Effectiveness Of PPO In Cooperative, Multi-agent Games (2021)0.00
• Breaking The Curse Of Multiagency In Robust Multi-agent Reinforcement Learning (2024)0.00
• A Generalized Training Approach For Multiagent Learning (2019)0.00
• Non-cooperative Multi-agent Systems With Exploring Agents (2020)0.00