Madiff: Offline Multi-agent Learning With Diffusion Models

Abstract

Offline reinforcement learning (RL) aims to learn policies from pre-existing datasets without further interactions, making it a challenging task. Q-learning algorithms struggle with extrapolation errors in offline settings, while supervised learning methods are constrained by model expressiveness. Recently, diffusion models (DMs) have shown promise in overcoming these limitations in single-agent learning, but their application in multi-agent scenarios remains unclear. Generating trajectories for each agent with independent DMs may impede coordination, while concatenating all agents' information can lead to low sample efficiency. Accordingly, we propose MADiff, which is realized with an attention-based diffusion model to model the complex coordination among behaviors of multiple agents. To our knowledge, MADiff is the first diffusion-based multi-agent learning framework, functioning as both a decentralized policy and a centralized controller. During decentralized executions, MADiff simu

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keyzhu2023madiff

Related papers

• Diffusion Models For Offline Multi-agent Reinforcement Learning With Safety Constraints (2024)0.00
• Preferred-action-optimized Diffusion Policies For Offline Reinforcement Learning (2024)0.00
• Long-horizon Rollout Via Dynamics Diffusion For Offline Reinforcement Learning (2024)1.81
• Beyond Conservatism: Diffusion Policies In Offline Multi-agent Reinforcement Learning (2023)0.00
• Diffusion Policies Creating A Trust Region For Offline Reinforcement Learning (2024)8.04
• Diffusion Policies As An Expressive Policy Class For Offline Reinforcement Learning (2022)0.00
• Policy Representation Via Diffusion Probability Model For Reinforcement Learning (2023)0.00
• Diffpogan: Diffusion Policies With Generative Adversarial Networks For Offline Reinforcement Learning (2024)0.00