Emergent Coordination Through Competition

Abstract

We study the emergence of cooperative behaviors in reinforcement learning agents by introducing a challenging competitive multi-agent soccer environment with continuous simulated physics. We demonstrate that decentralized, population-based training with co-play can lead to a progression in agents' behaviors: from random, to simple ball chasing, and finally showing evidence of cooperation. Our study highlights several of the challenges encountered in large scale multi-agent training in continuous control. In particular, we demonstrate that the automatic optimization of simple shaping rewards, not themselves conducive to co-operative behavior, can lead to long-horizon team behavior. We further apply an evaluation scheme, grounded by game theoretic principals, that can assess agent performance in the absence of pre-defined evaluation tasks or human baselines.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyliu2019emergent

Related papers

• Learning To Play Soccer From Scratch: Sample-efficient Emergent Coordination Through Curriculum-learning And Competition (2021)0.00
• Collaboration Of AI Agents Via Cooperative Multi-agent Deep Reinforcement Learning (2019)0.00
• Promoting Coordination Through Policy Regularization In Multi-agent Deep Reinforcement Learning (2019)0.00
• Coach-player Multi-agent Reinforcement Learning For Dynamic Team Composition (2021)0.00
• Multi-agent Actor-critic For Mixed Cooperative-competitive Environments (2017)0.00
• Emergent Cooperation Under Uncertain Incentive Alignment (2024)2.26
• Multi-agent Curricula And Emergent Implicit Signaling (2021)0.00
• Mimicking To Dominate: Imitation Learning Strategies For Success In Multiagent Competitive Games (2023)0.00