Learning Robust State Abstractions For Hidden-parameter Block Mdps

Abstract

Many control tasks exhibit similar dynamics that can be modeled as having common latent structure. Hidden-Parameter Markov Decision Processes (HiP-MDPs) explicitly model this structure to improve sample efficiency in multi-task settings. However, this setting makes strong assumptions on the observability of the state that limit its application in real-world scenarios with rich observation spaces. In this work, we leverage ideas of common structure from the HiP-MDP setting, and extend it to enable robust state abstractions inspired by Block MDPs. We derive instantiations of this new framework for both multi-task reinforcement learning (MTRL) and meta-reinforcement learning (Meta-RL) settings. Further, we provide transfer and generalization bounds based on task and state similarity, along with sample complexity bounds that depend on the aggregate number of samples across tasks, rather than the number of tasks, a significant improvement over prior work that use the same environment assump

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhang2020learning

Related papers

• Provable Benefits Of Multi-task RL Under Non-markovian Decision Making Processes (2023)0.00
• Learning Markov State Abstractions For Deep Reinforcement Learning (2021)0.00
• Group Distributionally Robust Reinforcement Learning With Hierarchical Latent Variables (2022)0.00
• Provable Multi-task Reinforcement Learning: A Representation Learning Framework With Low Rank Rewards (2026)0.00
• Asymptotically Optimal Reinforcement Learning In Block Markov Decision Processes (2025)0.00
• Invariant Causal Prediction For Block Mdps (2020)0.00
• On Learning History Based Policies For Controlling Markov Decision Processes (2022)0.00
• Low-dimensional State And Action Representation Learning With MDP Homomorphism Metrics (2021)0.00