Fully Distributed Actor-critic Architecture For Multitask Deep Reinforcement Learning

Abstract

We propose a fully distributed actor-critic architecture, named Diff-DAC, with application to multitask reinforcement learning (MRL). During the learning process, agents communicate their value and policy parameters to their neighbours, diffusing the information across a network of agents with no need for a central station. Each agent can only access data from its local task, but aims to learn a common policy that performs well for the whole set of tasks. The architecture is scalable, since the computational and communication cost per agent depends on the number of neighbours rather than the overall number of agents. We derive Diff-DAC from duality theory and provide novel insights into the actor-critic framework, showing that it is actually an instance of the dual ascent method. We prove almost sure convergence of Diff-DAC to a common policy under general assumptions that hold even for deep-neural network approximations. For more restrictive assumptions, we also prove that this common

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keymacua2021fully

Related papers

• Diff-dac: Distributed Actor-critic For Average Multitask Deep Reinforcement Learning (2017)0.00
• Distributional Soft Actor-critic With Diffusion Policy (2025)0.00
• A Multi-agent Off-policy Actor-critic Algorithm For Distributed Reinforcement Learning (2019)11.39
• Enhanced DACER Algorithm With High Diffusion Efficiency (2025)0.00
• Broad Critic Deep Actor Reinforcement Learning For Continuous Control (2024)0.00
• Decomposed Soft Actor-critic Method For Cooperative Multi-agent Reinforcement Learning (2021)0.00
• Decentralized Task Scheduling In Distributed Systems: A Deep Reinforcement Learning Approach (2026)0.00
• Natural Policy Gradient And Actor Critic Methods For Constrained Multi-task Reinforcement Learning (2024)0.00