Uncertainty-aware Transfer Across Tasks Using Hybrid Model-based Successor Feature Reinforcement Learning

Abstract

Sample efficiency is central to developing practical reinforcement learning (RL) for complex and large-scale decision-making problems. The ability to transfer and generalize knowledge gained from previous experiences to downstream tasks can significantly improve sample efficiency. Recent research indicates that successor feature (SF) RL algorithms enable knowledge generalization between tasks with different rewards but identical transition dynamics. It has recently been hypothesized that combining model-based (MB) methods with SF algorithms can alleviate the limitation of fixed transition dynamics. Furthermore, uncertainty-aware exploration is widely recognized as another appealing approach for improving sample efficiency. Putting together two ideas of hybrid model-based successor feature (MB-SF) and uncertainty leads to an approach to the problem of sample efficient uncertainty-aware knowledge transfer across tasks with different transition dynamics or/and reward functions. In this pa

Stats

• citations4
• S2 citations—
• github stars0
• HF likes0
• heat score5.24
• arxiv keymalekzadeh2023uncertainty

Related papers

• Self-supervised Reinforcement Learning That Transfers Using Random Features (2023)2.26
• SF-DQN: Provable Knowledge Transfer Using Successor Feature For Deep Reinforcement Learning (2024)0.00
• Hybrid Transfer Reinforcement Learning: Provable Sample Efficiency From Shifted-dynamics Data (2024)0.00
• Successor Feature Representations (2021)0.00
• Successor Features For Transfer In Reinforcement Learning (2016)0.00
• Advantages And Limitations Of Using Successor Features For Transfer In Reinforcement Learning (2017)0.00
• Transfer With Model Features In Reinforcement Learning (2018)0.00
• A New Representation Of Successor Features For Transfer Across Dissimilar Environments (2021)0.00