Provable RL With Exogenous Distractors Via Multistep Inverse Dynamics

Abstract

Many real-world applications of reinforcement learning (RL) require the agent to deal with high-dimensional observations such as those generated from a megapixel camera. Prior work has addressed such problems with representation learning, through which the agent can provably extract endogenous, latent state information from raw observations and subsequently plan efficiently. However, such approaches can fail in the presence of temporally correlated noise in the observations, a phenomenon that is common in practice. We initiate the formal study of latent state discovery in the presence of such exogenous noise sources by proposing a new model, the Exogenous Block MDP (EX-BMDP), for rich observation RL. We start by establishing several negative results, by highlighting failure cases of prior representation learning based approaches. Then, we introduce the Predictive Path Elimination (PPE) algorithm, that learns a generalization of inverse dynamics and is provably sample and computationall

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyefroni2021provable

Related papers

• Sample-efficient Reinforcement Learning In The Presence Of Exogenous Information (2022)0.00
• Offline Action-free Learning Of Ex-bmdps By Comparing Diverse Datasets (2025)0.00
• Asymptotically Optimal Reinforcement Learning In Block Markov Decision Processes (2025)0.00
• Extracting Latent State Representations With Linear Dynamics From Rich Observations (2020)0.00
• Learning A Fast Mixing Exogenous Block MDP Using A Single Trajectory (2024)0.00
• Multistep Inverse Is Not All You Need (2024)1.20
• Discovering And Removing Exogenous State Variables And Rewards For Reinforcement Learning (2018)0.00
• Deepmdp: Learning Continuous Latent Space Models For Representation Learning (2019)0.00