Agent-aware Training For Agent-agnostic Action Advising In Deep Reinforcement Learning

Abstract

Action advising endeavors to leverage supplementary guidance from expert teachers to alleviate the issue of sampling inefficiency in Deep Reinforcement Learning (DRL). Previous agent-specific action advising methods are hindered by imperfections in the agent itself, while agent-agnostic approaches exhibit limited adaptability to the learning agent. In this study, we propose a novel framework called Agent-Aware trAining yet Agent-Agnostic Action Advising (A7) to strike a balance between the two. The underlying concept of A7 revolves around utilizing the similarity of state features as an indicator for soliciting advice. However, unlike prior methodologies, the measurement of state feature similarity is performed by neither the error-prone learning agent nor the agent-agnostic advisor. Instead, we employ a proxy model to extract state features that are both discriminative (adaptive to the agent) and generally applicable (robust to agent noise). Furthermore, we utilize behavior cloning to

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keywei2023agent

Related papers

• Explainable Action Advising For Multi-agent Reinforcement Learning (2022)6.77
• Differential Advising In Multi-agent Reinforcement Learning (2020)0.00
• Multi-agent Advisor Q-learning (2021)0.00
• Learning From Multiple Independent Advisors In Multi-agent Reinforcement Learning (2023)0.00
• Introspective Action Advising For Interpretable Transfer Learning (2023)0.00
• Robust Deep Reinforcement Learning With Adaptive Adversarial Perturbations In Action Space (2024)6.20
• Ask-ac: An Initiative Advisor-in-the-loop Actor-critic Framework (2022)3.58
• Cautiously-optimistic Knowledge Sharing For Cooperative Multi-agent Reinforcement Learning (2023)5.84