Theoretically-grounded Policy Advice From Multiple Teachers In Reinforcement Learning Settings With Applications To Negative Transfer

Abstract

Policy advice is a transfer learning method where a student agent is able to learn faster via advice from a teacher. However, both this and other reinforcement learning transfer methods have little theoretical analysis. This paper formally defines a setting where multiple teacher agents can provide advice to a student and introduces an algorithm to leverage both autonomous exploration and teacher's advice. Our regret bounds justify the intuition that good teachers help while bad teachers hurt. Using our formalization, we are also able to quantify, for the first time, when negative transfer can occur within such a reinforcement learning setting.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhan2016theoretically

Related papers

• Introspective Action Advising For Interpretable Transfer Learning (2023)0.00
• Student/teacher Advising Through Reward Augmentation (2020)0.00
• Explainable Action Advising For Multi-agent Reinforcement Learning (2022)6.77
• Directed Policy Gradient For Safe Reinforcement Learning With Human Advice (2018)0.00
• Differential Advising In Multi-agent Reinforcement Learning (2020)0.00
• An Advantage Based Policy Transfer Algorithm For Reinforcement Learning With Measures Of Transferability (2023)0.00
• Improving Interactive Reinforcement Learning: What Makes A Good Teacher? (2019)11.19
• Agent-aware Training For Agent-agnostic Action Advising In Deep Reinforcement Learning (2023)2.26