Imagine, Initialize, And Explore: An Effective Exploration Method In Multi-agent Reinforcement Learning

Abstract

Effective exploration is crucial to discovering optimal strategies for multi-agent reinforcement learning (MARL) in complex coordination tasks. Existing methods mainly utilize intrinsic rewards to enable committed exploration or use role-based learning for decomposing joint action spaces instead of directly conducting a collective search in the entire action-observation space. However, they often face challenges obtaining specific joint action sequences to reach successful states in long-horizon tasks. To address this limitation, we propose Imagine, Initialize, and Explore (IIE), a novel method that offers a promising solution for efficient multi-agent exploration in complex scenarios. IIE employs a transformer model to imagine how the agents reach a critical state that can influence each other's transition functions. Then, we initialize the environment at this state using a simulator before the exploration phase. We formulate the imagination as a sequence modeling problem, where the s

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keyliu2024imagine

Related papers

• Individual Contributions As Intrinsic Exploration Scaffolds For Multi-agent Reinforcement Learning (2024)2.80
• AIR: Unifying Individual And Collective Exploration In Cooperative Multi-agent Reinforcement Learning (2024)0.00
• Coordinated Exploration Via Intrinsic Rewards For Multi-agent Reinforcement Learning (2019)0.00
• Prioritized Guidance For Efficient Multi-agent Reinforcement Learning Exploration (2019)0.00
• REMAX: Relational Representation For Multi-agent Exploration (2020)2.26
• Influence-based Reinforcement Learning For Intrinsically-motivated Agents (2021)0.00
• MIR: Efficient Exploration In Episodic Multi-agent Reinforcement Learning Via Mutual Intrinsic Reward (2025)0.00
• Episodic Multi-agent Reinforcement Learning With Curiosity-driven Exploration (2021)0.00