Improving Reinforcement Learning Efficiency With Auxiliary Tasks In Non-visual Environments: A Comparison

Abstract

Real-world reinforcement learning (RL) environments, whether in robotics or industrial settings, often involve non-visual observations and require not only efficient but also reliable and thus interpretable and flexible RL approaches. To improve efficiency, agents that perform state representation learning with auxiliary tasks have been widely studied in visual observation contexts. However, for real-world problems, dedicated representation learning modules that are decoupled from RL agents are more suited to meet requirements. This study compares common auxiliary tasks based on, to the best of our knowledge, the only decoupled representation learning method for low-dimensional non-visual observations. We evaluate potential improvements in sample efficiency and returns for environments ranging from a simple pendulum to a complex simulated robotics task. Our findings show that representation learning with auxiliary tasks only provides performance gains in sufficiently complex environmen

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keylange2023improving

Related papers

• Return-based Contrastive Representation Learning For Reinforcement Learning (2021)12.17
• Proto-value Networks: Scaling Representation Learning With Auxiliary Tasks (2023)0.00
• When Does Self-prediction Help? Understanding Auxiliary Tasks In Reinforcement Learning (2024)0.00
• On The Effect Of Auxiliary Tasks On Representation Dynamics (2021)0.00
• Ensemble And Auxiliary Tasks For Data-efficient Deep Reinforcement Learning (2021)0.00
• Human-inspired Framework To Accelerate Reinforcement Learning (2023)0.00
• Improving Sample Efficiency In Model-free Reinforcement Learning From Images (2019)16.99
• Auxiliary Task Discovery Through Generate-and-test (2022)0.00