Latent Variable Representation For Reinforcement Learning

Abstract

Deep latent variable models have achieved significant empirical successes in model-based reinforcement learning (RL) due to their expressiveness in modeling complex transition dynamics. On the other hand, it remains unclear theoretically and empirically how latent variable models may facilitate learning, planning, and exploration to improve the sample efficiency of RL. In this paper, we provide a representation view of the latent variable models for state-action value functions, which allows both tractable variational learning algorithm and effective implementation of the optimism/pessimism principle in the face of uncertainty for exploration. In particular, we propose a computationally efficient planning algorithm with UCB exploration by incorporating kernel embeddings of latent variable models. Theoretically, we establish the sample complexity of the proposed approach in the online and offline settings. Empirically, we demonstrate superior performance over current state-of-the-art al

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyren2022latent

Related papers

• Simplifying Model-based RL: Learning Representations, Latent-space Models, And Policies With One Objective (2022)0.00
• Reinforcement Learning Under Latent Dynamics: Toward Statistical And Algorithmic Modularity (2024)0.00
• Value-consistent Representation Learning For Data-efficient Reinforcement Learning (2022)0.00
• Extracting Latent State Representations With Linear Dynamics From Rich Observations (2020)0.00
• Learning Temporally-consistent Representations For Data-efficient Reinforcement Learning (2021)0.00
• Deepmdp: Learning Continuous Latent Space Models For Representation Learning (2019)0.00
• Provably Efficient Exploration For Reinforcement Learning Using Unsupervised Learning (2020)0.00
• Learning Dynamics Model In Reinforcement Learning By Incorporating The Long Term Future (2019)0.00