Transfer Heterogeneous Knowledge Among Peer-to-peer Teammates: A Model Distillation Approach

Abstract

Peer-to-peer knowledge transfer in distributed environments has emerged as a promising method since it could accelerate learning and improve team-wide performance without relying on pre-trained teachers in deep reinforcement learning. However, for traditional peer-to-peer methods such as action advising, they have encountered difficulties in how to efficiently expressed knowledge and advice. As a result, we propose a brand new solution to reuse experiences and transfer value functions among multiple students via model distillation. But it is still challenging to transfer Q-function directly since it is unstable and not bounded. To address this issue confronted with existing works, we adopt Categorical Deep Q-Network. We also describe how to design an efficient communication protocol to exploit heterogeneous knowledge among multiple distributed agents. Our proposed framework, namely Learning and Teaching Categorical Reinforcement (LTCR), shows promising performance on stabilizing and ac

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyxue2020transfer

Related papers

• Dual Policy Distillation (2020)10.61
• Goal-conditioned Q-learning As Knowledge Distillation (2022)3.58
• Continual Policy Distillation From Distributed Reinforcement Learning Teachers (2026)0.00
• Knowru: Knowledge Reusing Via Knowledge Distillation In Multi-agent Reinforcement Learning (2021)9.23
• Improved Reinforcement Learning In Cooperative Multi-agent Environments Using Knowledge Transfer (2021)0.00
• Student/teacher Advising Through Reward Augmentation (2020)0.00
• Learning To Teach In Cooperative Multiagent Reinforcement Learning (2018)0.00
• Online Policy Distillation With Decision-attention (2024)0.00