A Model-based Solution To The Offline Multi-agent Reinforcement Learning Coordination Problem

Abstract

Training multiple agents to coordinate is an essential problem with applications in robotics, game theory, economics, and social sciences. However, most existing Multi-Agent Reinforcement Learning (MARL) methods are online and thus impractical for real-world applications in which collecting new interactions is costly or dangerous. While these algorithms should leverage offline data when available, doing so gives rise to what we call the offline coordination problem. Specifically, we identify and formalize the strategy agreement (SA) and the strategy fine-tuning (SFT) coordination challenges, two issues at which current offline MARL algorithms fail. Concretely, we reveal that the prevalent model-free methods are severely deficient and cannot handle coordination-intensive offline multi-agent tasks in either toy or MuJoCo domains. To address this setback, we emphasize the importance of inter-agent interactions and propose the very first model-based offline MARL method. Our resulting algor

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keybarde2023a

Related papers

• Coordination-driven Learning In Multi-agent Problem Spaces (2018)0.00
• Offline Multi-agent Reinforcement Learning Via In-sample Sequential Policy Optimization (2024)0.00
• Learning To Coordinate In Multi-agent Systems: A Coordinated Actor-critic Algorithm And Finite-time Guarantees (2021)0.00
• Off-the-grid MARL: Datasets With Baselines For Offline Multi-agent Reinforcement Learning (2023)2.26
• Incentivize Without Bonus: Provably Efficient Model-based Online Multi-agent RL For Markov Games (2025)0.00
• Robust Multi-agent Reinforcement Learning With Social Empowerment For Coordination And Communication (2020)0.00
• Learning From Good Trajectories In Offline Multi-agent Reinforcement Learning (2022)5.24
• Learning To Advise And Learning From Advice In Cooperative Multi-agent Reinforcement Learning (2022)0.00