Learning From Multiple Independent Advisors In Multi-agent Reinforcement Learning

Abstract

Multi-agent reinforcement learning typically suffers from the problem of sample inefficiency, where learning suitable policies involves the use of many data samples. Learning from external demonstrators is a possible solution that mitigates this problem. However, most prior approaches in this area assume the presence of a single demonstrator. Leveraging multiple knowledge sources (i.e., advisors) with expertise in distinct aspects of the environment could substantially speed up learning in complex environments. This paper considers the problem of simultaneously learning from multiple independent advisors in multi-agent reinforcement learning. The approach leverages a two-level Q-learning architecture, and extends this framework from single-agent to multi-agent settings. We provide principled algorithms that incorporate a set of advisors by both evaluating the advisors at each state and subsequently using the advisors to guide action selection. We also provide theoretical convergence an

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keysubramanian2023learning

Related papers

• Multi-agent Advisor Q-learning (2021)0.00
• Differential Advising In Multi-agent Reinforcement Learning (2020)0.00
• Learning To Advise And Learning From Advice In Cooperative Multi-agent Reinforcement Learning (2022)0.00
• Individual Specialization In Multi-task Environments With Multiagent Reinforcement Learners (2019)0.00
• Cautiously-optimistic Knowledge Sharing For Cooperative Multi-agent Reinforcement Learning (2023)5.84
• Emergent Social Learning Via Multi-agent Reinforcement Learning (2020)0.00
• Explainable Action Advising For Multi-agent Reinforcement Learning (2022)6.77
• Deep Multiagent Reinforcement Learning: Challenges And Directions (2021)0.00