Model-free Representation Learning And Exploration In Low-rank Mdps

Abstract

The low rank MDP has emerged as an important model for studying representation learning and exploration in reinforcement learning. With a known representation, several model-free exploration strategies exist. In contrast, all algorithms for the unknown representation setting are model-based, thereby requiring the ability to model the full dynamics. In this work, we present the first model-free representation learning algorithms for low rank MDPs. The key algorithmic contribution is a new minimax representation learning objective, for which we provide variants with differing tradeoffs in their statistical and computational properties. We interleave this representation learning step with an exploration strategy to cover the state space in a reward-free manner. The resulting algorithms are provably sample efficient and can accommodate general function approximation to scale to complex environments.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymodi2021model

Related papers

• Improved Sample Complexity For Reward-free Reinforcement Learning Under Low-rank Mdps (2023)0.00
• Optimal Horizon-free Reward-free Exploration For Linear Mixture Mdps (2023)0.00
• Low-dimensional State And Action Representation Learning With MDP Homomorphism Metrics (2021)0.00
• Model-based Exploration In Monitored Markov Decision Processes (2025)0.00
• Sample-efficient Reinforcement Learning Is Feasible For Linearly Realizable Mdps With Limited Revisiting (2021)0.00
• Model-free Reinforcement Learning In Infinite-horizon Average-reward Markov Decision Processes (2019)0.00
• Improved Bounds For Reward-agnostic And Reward-free Exploration (2026)0.00
• Reward-free Model-based Reinforcement Learning With Linear Function Approximation (2021)0.00