Event-based Communication In Distributed Q-learning

Abstract

We present an approach to reduce the communication of information needed on a Distributed Q-Learning system inspired by Event Triggered Control (ETC) techniques. We consider a baseline scenario of a distributed Q-learning problem on a Markov Decision Process (MDP). Following an event-based approach, N agents explore the MDP and communicate experiences to a central learner only when necessary, which performs updates of the actor Q functions. We design an Event Based distributed Q learning system (EBd-Q), and derive convergence guarantees with respect to a vanilla Q-learning algorithm. We present experimental results showing that event-based communication results in a substantial reduction of data transmission rates in such distributed systems. Additionally, we discuss what effects (desired and undesired) these event-based approaches have on the learning processes studied, and how they can be applied to more complex multi-agent systems.

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keyornia2021event

Related papers

• Event-triggered Multi-agent Reinforcement Learning With Communication Under Limited-bandwidth Constraint (2020)0.00
• Robust Event-driven Interactions In Cooperative Multi-agent Learning (2022)0.00
• Federated Q-learning: Linear Regret Speedup With Low Communication Cost (2023)0.00
• Provably Efficient Multi-agent Reinforcement Learning With Fully Decentralized Communication (2021)0.00
• Communication-efficient Policy Gradient Methods For Distributed Reinforcement Learning (2018)13.05
• Distributed Q-learning With State Tracking For Multi-agent Networked Control (2020)0.00
• Robust Event-triggered Integrated Communication And Control With Graph Information Bottleneck Optimization (2025)0.00
• A Multi-agent Multi-environment Mixed Q-learning For Partially Decentralized Wireless Network Optimization (2024)0.00