MANSA: Learning Fast And Slow In Multi-agent Systems

Abstract

In multi-agent reinforcement learning (MARL), independent learning (IL) often shows remarkable performance and easily scales with the number of agents. Yet, using IL can be inefficient and runs the risk of failing to successfully train, particularly in scenarios that require agents to coordinate their actions. Using centralised learning (CL) enables MARL agents to quickly learn how to coordinate their behaviour but employing CL everywhere is often prohibitively expensive in real-world applications. Besides, using CL in value-based methods often needs strong representational constraints (e.g. individual-global-max condition) that can lead to poor performance if violated. In this paper, we introduce a novel plug & play IL framework named Multi-Agent Network Selection Algorithm (MANSA) which selectively employs CL only at states that require coordination. At its core, MANSA has an additional agent that uses switching controls to quickly learn the best states to activate CL during training

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymguni2023mansa

Related papers

• Mean-field Multi-agent Reinforcement Learning: A Decentralized Network Approach (2021)0.00
• Marllib: A Scalable And Efficient Multi-agent Reinforcement Learning Library (2022)0.00
• Dealing With Non-stationarity In Decentralized Cooperative Multi-agent Deep Reinforcement Learning Via Multi-timescale Learning (2023)0.00
• MARL-LNS: Cooperative Multi-agent Reinforcement Learning Via Large Neighborhoods Search (2024)0.00
• Local Advantage Networks For Cooperative Multi-agent Reinforcement Learning (2021)0.00
• Characterizing Speed Performance Of Multi-agent Reinforcement Learning (2023)4.52
• An Initial Introduction To Cooperative Multi-agent Reinforcement Learning (2024)0.00
• Learning In Cooperative Multiagent Systems Using Cognitive And Machine Models (2023)7.81