Communication-efficient Consensus Mechanism For Federated Reinforcement Learning

Abstract

The paper considers independent reinforcement learning (IRL) for multi-agent decision-making process in the paradigm of federated learning (FL). We show that FL can clearly improve the policy performance of IRL in terms of training efficiency and stability. However, since the policy parameters are trained locally and aggregated iteratively through a central server in FL, frequent information exchange incurs a large amount of communication overheads. To reach a good balance between improving the model's convergence performance and reducing the required communication and computation overheads, this paper proposes a system utility function and develops a consensus-based optimization scheme on top of the periodic averaging method, which introduces the consensus algorithm into FL for the exchange of a model's local gradients. This paper also provides novel convergence guarantees for the developed method, and demonstrates its superior effectiveness and efficiency in improving the system util

Stats

• citations7
• S2 citations—
• github stars0
• HF likes0
• heat score6.77
• arxiv keyxu2022communication

Related papers

• The Gradient Convergence Bound Of Federated Multi-agent Reinforcement Learning With Efficient Communication (2021)0.00
• Optimized Local Updates In Federated Learning Via Reinforcement Learning (2025)0.00
• Federated Ensemble Model-based Reinforcement Learning In Edge Computing (2021)11.58
• A Multi-agent Reinforcement Learning Approach For Efficient Client Selection In Federated Learning (2022)15.00
• A Fair Federated Learning Framework With Reinforcement Learning (2022)0.00
• Momentum-based Federated Reinforcement Learning With Interaction And Communication Efficiency (2024)4.52
• Momentum For The Win: Collaborative Federated Reinforcement Learning Across Heterogeneous Environments (2024)0.00
• Communication-efficient Policy Gradient Methods For Distributed Reinforcement Learning (2018)13.05