Multi-agent Advisor Q-learning

Abstract

In the last decade, there have been significant advances in multi-agent reinforcement learning (MARL) but there are still numerous challenges, such as high sample complexity and slow convergence to stable policies, that need to be overcome before wide-spread deployment is possible. However, many real-world environments already, in practice, deploy sub-optimal or heuristic approaches for generating policies. An interesting question that arises is how to best use such approaches as advisors to help improve reinforcement learning in multi-agent domains. In this paper, we provide a principled framework for incorporating action recommendations from online sub-optimal advisors in multi-agent settings. We describe the problem of ADvising Multiple Intelligent Reinforcement Agents (ADMIRAL) in nonrestrictive general-sum stochastic game environments and present two novel Q-learning based algorithms: ADMIRAL - Decision Making (ADMIRAL-DM) and ADMIRAL - Advisor Evaluation (ADMIRAL-AE), which allow

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keysubramanian2021multi

Related papers

• Learning From Multiple Independent Advisors In Multi-agent Reinforcement Learning (2023)0.00
• Learning To Advise And Learning From Advice In Cooperative Multi-agent Reinforcement Learning (2022)0.00
• ACE: Cooperative Multi-agent Q-learning With Bidirectional Action-dependency (2022)0.00
• Differential Advising In Multi-agent Reinforcement Learning (2020)0.00
• Explainable Action Advising For Multi-agent Reinforcement Learning (2022)6.77
• Agent-aware Training For Agent-agnostic Action Advising In Deep Reinforcement Learning (2023)2.26
• Multi-agent Actor-critic For Mixed Cooperative-competitive Environments (2017)0.00
• Mitigating Relative Over-generalization In Multi-agent Reinforcement Learning (2024)0.00