Laser Learning Environment: A New Environment For Coordination-critical Multi-agent Tasks

Abstract

We introduce the Laser Learning Environment (LLE), a collaborative multi-agent reinforcement learning environment in which coordination is central. In LLE, agents depend on each other to make progress (interdependence), must jointly take specific sequences of actions to succeed (perfect coordination), and accomplishing those joint actions does not yield any intermediate reward (zero-incentive dynamics). The challenge of such problems lies in the difficulty of escaping state space bottlenecks caused by interdependence steps since escaping those bottlenecks is not rewarded. We test multiple state-of-the-art value-based MARL algorithms against LLE and show that they consistently fail at the collaborative task because of their inability to escape state space bottlenecks, even though they successfully achieve perfect coordination. We show that Q-learning extensions such as prioritized experience replay and n-steps return hinder exploration in environments with zero-incentive dynamics, and f

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymolinghen2024laser

Related papers

• Coordination-driven Learning In Multi-agent Problem Spaces (2018)0.00
• Learning Multiagent Coordination In The Absence Of Communication Channels (2018)0.00
• Continuous Coordination As A Realistic Scenario For Lifelong Learning (2021)0.00
• Decentralized Multi-agent Reinforcement Learning Algorithm Using A Cluster-synchronized Laser Network (2024)0.00
• Learning To Advise And Learning From Advice In Cooperative Multi-agent Reinforcement Learning (2022)0.00
• Individual Specialization In Multi-task Environments With Multiagent Reinforcement Learners (2019)0.00
• LIGS: Learnable Intrinsic-reward Generation Selection For Multi-agent Learning (2021)0.00
• Locality Matters: A Scalable Value Decomposition Approach For Cooperative Multi-agent Reinforcement Learning (2021)0.00