Provably Efficient Multi-agent Reinforcement Learning With Fully Decentralized Communication

Abstract

A challenge in reinforcement learning (RL) is minimizing the cost of sampling associated with exploration. Distributed exploration reduces sampling complexity in multi-agent RL (MARL). We investigate the benefits to performance in MARL when exploration is fully decentralized. Specifically, we consider a class of online, episodic, tabular \(Q\)-learning problems under time-varying reward and transition dynamics, in which agents can communicate in a decentralized manner.We show that group performance, as measured by the bound on regret, can be significantly improved through communication when each agent uses a decentralized message-passing protocol, even when limited to sending information up to its \(\gamma\)-hop neighbors. We prove regret and sample complexity bounds that depend on the number of agents, communication network structure and \(\gamma.\) We show that incorporating more agents and more information sharing into the group learning scheme speeds up convergence to the optimal p

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylidard2021provably

Related papers

• Fully Decentralized Multi-agent Reinforcement Learning With Networked Agents (2018)0.00
• On Improving Model-free Algorithms For Decentralized Multi-agent Reinforcement Learning (2021)0.00
• Cooperative Multi-agent RL With Communication Constraints (2026)0.00
• Contextual Knowledge Sharing In Multi-agent Reinforcement Learning With Decentralized Communication And Coordination (2025)0.00
• Graph Exploration For Effective Multi-agent Q-learning (2023)5.24
• A Finite Time Analysis Of Distributed Q-learning (2024)0.00
• Mean-field Multi-agent Reinforcement Learning: A Decentralized Network Approach (2021)0.00
• Decentralized Multi-agent Reinforcement Learning With Networked Agents: Recent Advances (2019)0.00