On The Effect Of Auxiliary Tasks On Representation Dynamics

Abstract

While auxiliary tasks play a key role in shaping the representations learnt by reinforcement learning agents, much is still unknown about the mechanisms through which this is achieved. This work develops our understanding of the relationship between auxiliary tasks, environment structure, and representations by analysing the dynamics of temporal difference algorithms. Through this approach, we establish a connection between the spectral decomposition of the transition operator and the representations induced by a variety of auxiliary tasks. We then leverage insights from these theoretical results to inform the selection of auxiliary tasks for deep reinforcement learning agents in sparse-reward environments.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylyle2021on

Related papers

• When Does Self-prediction Help? Understanding Auxiliary Tasks In Reinforcement Learning (2024)0.00
• Work In Progress: Temporally Extended Auxiliary Tasks (2020)0.00
• Proto-value Networks: Scaling Representation Learning With Auxiliary Tasks (2023)0.00
• Improving Reinforcement Learning Efficiency With Auxiliary Tasks In Non-visual Environments: A Comparison (2023)2.26
• What Makes Useful Auxiliary Tasks In Reinforcement Learning: Investigating The Effect Of The Target Policy (2022)0.00
• Ensemble And Auxiliary Tasks For Data-efficient Deep Reinforcement Learning (2021)0.00
• Auxiliary Task Discovery Through Generate-and-test (2022)0.00
• Return-based Contrastive Representation Learning For Reinforcement Learning (2021)12.17