Introspective Action Advising For Interpretable Transfer Learning

Abstract

Transfer learning can be applied in deep reinforcement learning to accelerate the training of a policy in a target task by transferring knowledge from a policy learned in a related source task. This is commonly achieved by copying pretrained weights from the source policy to the target policy prior to training, under the constraint that they use the same model architecture. However, not only does this require a robust representation learned over a wide distribution of states -- often failing to transfer between specialist models trained over single tasks -- but it is largely uninterpretable and provides little indication of what knowledge is transferred. In this work, we propose an alternative approach to transfer learning between tasks based on action advising, in which a teacher trained in a source task actively guides a student's exploration in a target task. Through introspection, the teacher is capable of identifying when advice is beneficial to the student and should be given, an

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keycampbell2023introspective

Related papers

• Explainable Action Advising For Multi-agent Reinforcement Learning (2022)6.77
• Theoretically-grounded Policy Advice From Multiple Teachers In Reinforcement Learning Settings With Applications To Negative Transfer (2016)0.00
• Agent-aware Training For Agent-agnostic Action Advising In Deep Reinforcement Learning (2023)2.26
• Student/teacher Advising Through Reward Augmentation (2020)0.00
• An Advantage Based Policy Transfer Algorithm For Reinforcement Learning With Measures Of Transferability (2023)0.00
• Contextual Policy Transfer In Reinforcement Learning Domains Via Deep Mixtures-of-experts (2020)0.00
• IOB: Integrating Optimization Transfer And Behavior Transfer For Multi-policy Reuse (2023)5.24
• REPAINT: Knowledge Transfer In Deep Reinforcement Learning (2020)0.00