Diff-dac: Distributed Actor-critic For Average Multitask Deep Reinforcement Learning

Abstract

We propose a fully distributed actor-critic algorithm approximated by deep neural networks, named \textit\{Diff-DAC\}, with application to single-task and to average multitask reinforcement learning (MRL). Each agent has access to data from its local task only, but it aims to learn a policy that performs well on average for the whole set of tasks. During the learning process, agents communicate their value-policy parameters to their neighbors, diffusing the information across the network, so that they converge to a common policy, with no need for a central node. The method is scalable, since the computational and communication costs per agent grow with its number of neighbors. We derive Diff-DAC's from duality theory and provide novel insights into the standard actor-critic framework, showing that it is actually an instance of the dual ascent method that approximates the solution of a linear program. Experiments suggest that Diff-DAC can outperform the single previous distributed MRL a

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymacua2017diff

Related papers

• Fully Distributed Actor-critic Architecture For Multitask Deep Reinforcement Learning (2021)3.58
• Enhanced DACER Algorithm With High Diffusion Efficiency (2025)0.00
• Distributional Soft Actor-critic With Diffusion Policy (2025)0.00
• DR-SAC: Distributionally Robust Soft Actor-critic For Reinforcement Learning Under Uncertainty (2025)0.00
• Diffusion Actor-critic: Formulating Constrained Policy Iteration As Diffusion Noise Regression For Offline Reinforcement Learning (2024)2.92
• F2A2: Flexible Fully-decentralized Approximate Actor-critic For Cooperative Multi-agent Reinforcement Learning (2020)0.00
• Scalable Multi-agent Reinforcement Learning For Networked Systems With Average Reward (2020)0.00
• Broad Critic Deep Actor Reinforcement Learning For Continuous Control (2024)0.00