Policy Distillation And Value Matching In Multiagent Reinforcement Learning

Abstract

Multiagent reinforcement learning algorithms (MARL) have been demonstrated on complex tasks that require the coordination of a team of multiple agents to complete. Existing works have focused on sharing information between agents via centralized critics to stabilize learning or through communication to increase performance, but do not generally look at how information can be shared between agents to address the curse of dimensionality in MARL. We posit that a multiagent problem can be decomposed into a multi-task problem where each agent explores a subset of the state space instead of exploring the entire state space. This paper introduces a multiagent actor-critic algorithm and method for combining knowledge from homogeneous agents through distillation and value-matching that outperforms policy distillation alone and allows further learning in both discrete and continuous action spaces.

Stats

• citations24
• S2 citations—
• github stars0
• HF likes0
• heat score10.48
• arxiv keywadhwania2019policy

Related papers

• Modeling The Interaction Between Agents In Cooperative Multi-agent Reinforcement Learning (2021)0.00
• Revisiting Some Common Practices In Cooperative Multi-agent Reinforcement Learning (2022)0.00
• Individual Specialization In Multi-task Environments With Multiagent Reinforcement Learners (2019)0.00
• Benchmarking Multi-agent Deep Reinforcement Learning Algorithms In Cooperative Tasks (2020)0.00
• More Centralized Training, Still Decentralized Execution: Multi-agent Conditional Policy Factorization (2022)0.00
• Towards Global Optimality In Cooperative MARL With The Transformation And Distillation Framework (2022)0.00
• KD-MARL: Resource-aware Knowledge Distillation In Multi-agent Reinforcement Learning (2026)0.00
• Inducing Cooperation Via Team Regret Minimization Based Multi-agent Deep Reinforcement Learning (2019)0.00