Emergence Of Collective Open-ended Exploration From Decentralized Meta-reinforcement Learning

Abstract

Recent works have proven that intricate cooperative behaviors can emerge in agents trained using meta reinforcement learning on open ended task distributions using self-play. While the results are impressive, we argue that self-play and other centralized training techniques do not accurately reflect how general collective exploration strategies emerge in the natural world: through decentralized training and over an open-ended distribution of tasks. In this work we therefore investigate the emergence of collective exploration strategies, where several agents meta-learn independent recurrent policies on an open ended distribution of tasks. To this end we introduce a novel environment with an open ended procedurally generated task space which dynamically combines multiple subtasks sampled from five diverse task types to form a vast distribution of task trees. We show that decentralized agents trained in our environment exhibit strong generalization abilities when confronted with novel obj

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keybornemann2023emergence

Related papers

• Procedural Generation Of Meta-reinforcement Learning Tasks (2023)0.00
• Open-ended Learning Leads To Generally Capable Agents (2021)0.00
• From Centralized To Self-supervised: Pursuing Realistic Multi-agent Reinforcement Learning (2023)0.00
• MAESTRO: Open-ended Environment Design For Multi-agent Reinforcement Learning (2023)0.00
• Optimistic {\epsilon}-greedy Exploration For Cooperative Multi-agent Reinforcement Learning (2025)0.00
• Fully Decentralized Cooperative Multi-agent Reinforcement Learning: A Survey (2024)0.00
• Decoupling Exploration And Exploitation For Meta-reinforcement Learning Without Sacrifices (2020)0.00
• MESA: Cooperative Meta-exploration In Multi-agent Learning Through Exploiting State-action Space Structure (2024)2.26