Transfer RL Across Observation Feature Spaces Via Model-based Regularization

Abstract

In many reinforcement learning (RL) applications, the observation space is specified by human developers and restricted by physical realizations, and may thus be subject to dramatic changes over time (e.g. increased number of observable features). However, when the observation space changes, the previous policy will likely fail due to the mismatch of input features, and another policy must be trained from scratch, which is inefficient in terms of computation and sample complexity. Following theoretical insights, we propose a novel algorithm which extracts the latent-space dynamics in the source task, and transfers the dynamics model to the target task to use as a model-based regularizer. Our algorithm works for drastic changes of observation space (e.g. from vector-based observation to image-based observation), without any inter-task mapping or any prior knowledge of the target task. Empirical results show that our algorithm significantly improves the efficiency and stability of learni

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keysun2022transfer

Related papers

• Self-supervised Reinforcement Learning That Transfers Using Random Features (2023)2.26
• On The Feasibility Of Cross-task Transfer With Model-based Reinforcement Learning (2022)0.00
• Decoupling Regularization From The Action Space (2024)0.00
• Regularization Matters In Policy Optimization (2019)2.68
• Learning Adaptive Exploration Strategies In Dynamic Environments Through Informed Policy Regularization (2020)0.00
• Simplifying Model-based RL: Learning Representations, Latent-space Models, And Policies With One Objective (2022)0.00
• An Advantage Based Policy Transfer Algorithm For Reinforcement Learning With Measures Of Transferability (2023)0.00
• Reinforcement Learning In Feature Space: Matrix Bandit, Kernels, And Regret Bound (2019)0.00