Scaling Distributed Multi-task Reinforcement Learning With Experience Sharing

Abstract

Recently, DARPA launched the ShELL program, which aims to explore how experience sharing can benefit distributed lifelong learning agents in adapting to new challenges. In this paper, we address this issue by conducting both theoretical and empirical research on distributed multi-task reinforcement learning (RL), where a group of \(N\) agents collaboratively solves \(M\) tasks without prior knowledge of their identities. We approach the problem by formulating it as linearly parameterized contextual Markov decision processes (MDPs), where each task is represented by a context that specifies the transition dynamics and rewards. To tackle this problem, we propose an algorithm called DistMT-LSVI. First, the agents identify the tasks, and then they exchange information through a central server to derive \(\epsilon\)-optimal policies for the tasks. Our research demonstrates that to achieve \(\epsilon\)-optimal policies for all \(M\) tasks, a single agent using DistMT-LSVI needs to run a tota

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyamani2023scaling

Related papers

• Deep Decentralized Multi-task Multi-agent Reinforcement Learning Under Partial Observability (2017)0.00
• Learning To Share In Multi-agent Reinforcement Learning (2021)0.00
• LDSA: Learning Dynamic Subtask Assignment In Cooperative Multi-agent Reinforcement Learning (2022)3.58
• Individual Specialization In Multi-task Environments With Multiagent Reinforcement Learners (2019)0.00
• Policy Distillation And Value Matching In Multiagent Reinforcement Learning (2019)10.48
• Contextual Knowledge Sharing In Multi-agent Reinforcement Learning With Decentralized Communication And Coordination (2025)0.00
• Multi-task Multi-agent Shared Layers Are Universal Cognition Of Multi-agent Coordination (2023)0.00
• Provably Efficient Multi-task Reinforcement Learning With Model Transfer (2021)0.00