Learning And Understanding A Disentangled Feature Representation For Hidden Parameters In Reinforcement Learning

Abstract

Hidden parameters are latent variables in reinforcement learning (RL) environments that are constant over the course of a trajectory. Understanding what, if any, hidden parameters affect a particular environment can aid both the development and appropriate usage of RL systems. We present an unsupervised method to map RL trajectories into a feature space where distance represents the relative difference in system behavior due to hidden parameters. Our approach disentangles the effects of hidden parameters by leveraging a recurrent neural network (RNN) world model as used in model-based RL. First, we alter the standard world model training algorithm to isolate the hidden parameter information in the world model memory. Then, we use a metric learning approach to map the RNN memory into a space with a distance metric approximating a bisimulation metric with respect to the hidden parameters. The resulting disentangled feature space can be used to meaningfully relate trajectories to each oth

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyreale2022learning

Related papers

• On The Importance Of Feature Decorrelation For Unsupervised Representation Learning In Reinforcement Learning (2023)0.00
• Estimating Disentangled Belief About Hidden State And Hidden Task For Meta-rl (2021)0.00
• Conditional Mutual Information For Disentangled Representations In Reinforcement Learning (2023)0.00
• Parameter Space Noise For Exploration (2017)0.00
• Deepmdp: Learning Continuous Latent Space Models For Representation Learning (2019)0.00
• Latent Variable Representation For Reinforcement Learning (2022)0.00
• Learning Representations In Model-free Hierarchical Reinforcement Learning (2018)11.49
• Unsupervised Representation Learning In Deep Reinforcement Learning: A Review (2022)9.59