What Makes Useful Auxiliary Tasks In Reinforcement Learning: Investigating The Effect Of The Target Policy

Abstract

Auxiliary tasks have been argued to be useful for representation learning in reinforcement learning. Although many auxiliary tasks have been empirically shown to be effective for accelerating learning on the main task, it is not yet clear what makes useful auxiliary tasks. Some of the most promising results are on the pixel control, reward prediction, and the next state prediction auxiliary tasks; however, the empirical results are mixed, showing substantial improvements in some cases and marginal improvements in others. Careful investigations of how auxiliary tasks help the learning of the main task is necessary. In this paper, we take a step studying the effect of the target policies on the usefulness of the auxiliary tasks formulated as general value functions. General value functions consist of three core elements: 1) policy 2) cumulant 3) continuation function. Our focus on the role of the target policy of the auxiliary tasks is motivated by the fact that the target policy determi

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyrafiee2022what

Related papers

• Proto-value Networks: Scaling Representation Learning With Auxiliary Tasks (2023)0.00
• Continual Auxiliary Task Learning (2022)0.00
• Work In Progress: Temporally Extended Auxiliary Tasks (2020)0.00
• Auxiliary Task Discovery Through Generate-and-test (2022)0.00
• When Does Self-prediction Help? Understanding Auxiliary Tasks In Reinforcement Learning (2024)0.00
• On The Effect Of Auxiliary Tasks On Representation Dynamics (2021)0.00
• Improving Reinforcement Learning Efficiency With Auxiliary Tasks In Non-visual Environments: A Comparison (2023)2.26
• Ensemble And Auxiliary Tasks For Data-efficient Deep Reinforcement Learning (2021)0.00