Multistep Inverse Is Not All You Need

Abstract

In real-world control settings, the observation space is often unnecessarily high-dimensional and subject to time-correlated noise. However, the controllable dynamics of the system are often far simpler than the dynamics of the raw observations. It is therefore desirable to learn an encoder to map the observation space to a simpler space of control-relevant variables. In this work, we consider the Ex-BMDP model, first proposed by Efroni et al. (2022), which formalizes control problems where observations can be factorized into an action-dependent latent state which evolves deterministically, and action-independent time-correlated noise. Lamb et al. (2022) proposes the "AC-State" method for learning an encoder to extract a complete action-dependent latent state representation from the observations in such problems. AC-State is a multistep-inverse method, in that it uses the encoding of the the first and last state in a path to predict the first action in the path. However, we identify ca

Stats

• citations0
• S2 citations—
• github stars3
• HF likes0
• heat score1.20
• arxiv keylevine2024multistep

Related papers

• Provable RL With Exogenous Distractors Via Multistep Inverse Dynamics (2021)0.00
• Offline Action-free Learning Of Ex-bmdps By Comparing Diverse Datasets (2025)0.00
• Generalizing Multi-step Inverse Models For Representation Learning To Finite-memory Pomdps (2024)0.00
• Learning A Fast Mixing Exogenous Block MDP Using A Single Trajectory (2024)0.00
• Probabilistic Inverse Optimal Control For Non-linear Partially Observable Systems Disentangles Perceptual Uncertainty And Behavioral Costs (2023)0.00
• Deep Inverse Q-learning With Constraints (2020)0.00
• Inverse Rational Control With Partially Observable Continuous Nonlinear Dynamics (2019)0.00
• Multi-action Restless Bandits With Weakly Coupled Constraints: Simultaneous Learning And Control (2024)0.00