Diffusion-based Episodes Augmentation For Offline Multi-agent Reinforcement Learning

Abstract

Offline multi-agent reinforcement learning (MARL) is increasingly recognized as crucial for effectively deploying RL algorithms in environments where real-time interaction is impractical, risky, or costly. In the offline setting, learning from a static dataset of past interactions allows for the development of robust and safe policies without the need for live data collection, which can be fraught with challenges. Building on this foundational importance, we present EAQ, Episodes Augmentation guided by Q-total loss, a novel approach for offline MARL framework utilizing diffusion models. EAQ integrates the Q-total function directly into the diffusion model as a guidance to maximize the global returns in an episode, eliminating the need for separate training. Our focus primarily lies on cooperative scenarios, where agents are required to act collectively towards achieving a shared goal-essentially, maximizing global returns. Consequently, we demonstrate that our episodes augmentation in

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyoh2024diffusion

Related papers

• Diffusion Models For Offline Multi-agent Reinforcement Learning With Safety Constraints (2024)0.00
• Diffusion Policies Creating A Trust Region For Offline Reinforcement Learning (2024)8.04
• Madiff: Offline Multi-agent Learning With Diffusion Models (2023)2.26
• DIAR: Diffusion-model-guided Implicit Q-learning With Adaptive Revaluation (2024)0.00
• Diffusion Policies As An Expressive Policy Class For Offline Reinforcement Learning (2022)0.00
• Beyond Conservatism: Diffusion Policies In Offline Multi-agent Reinforcement Learning (2023)0.00
• Entropy-regularized Diffusion Policy With Q-ensembles For Offline Reinforcement Learning (2024)3.58
• Preferred-action-optimized Diffusion Policies For Offline Reinforcement Learning (2024)0.00