Explainable Action Advising For Multi-agent Reinforcement Learning

Abstract

Action advising is a knowledge transfer technique for reinforcement learning based on the teacher-student paradigm. An expert teacher provides advice to a student during training in order to improve the student's sample efficiency and policy performance. Such advice is commonly given in the form of state-action pairs. However, it makes it difficult for the student to reason with and apply to novel states. We introduce Explainable Action Advising, in which the teacher provides action advice as well as associated explanations indicating why the action was chosen. This allows the student to self-reflect on what it has learned, enabling advice generalization and leading to improved sample efficiency and learning performance - even in environments where the teacher is sub-optimal. We empirically show that our framework is effective in both single-agent and multi-agent scenarios, yielding improved policy returns and convergence rates when compared to state-of-the-art methods

Stats

• citations7
• S2 citations—
• github stars0
• HF likes0
• heat score6.77
• arxiv keyguo2022explainable

Related papers

• Agent-aware Training For Agent-agnostic Action Advising In Deep Reinforcement Learning (2023)2.26
• Introspective Action Advising For Interpretable Transfer Learning (2023)0.00
• Differential Advising In Multi-agent Reinforcement Learning (2020)0.00
• Multi-agent Advisor Q-learning (2021)0.00
• Learning From Multiple Independent Advisors In Multi-agent Reinforcement Learning (2023)0.00
• Theoretically-grounded Policy Advice From Multiple Teachers In Reinforcement Learning Settings With Applications To Negative Transfer (2016)0.00
• Ask-ac: An Initiative Advisor-in-the-loop Actor-critic Framework (2022)3.58
• Peer Learning: Learning Complex Policies In Groups From Scratch Via Action Recommendations (2023)0.00