Imagine Beyond! Distributionally Robust Auto-encoding For State Space Coverage In Online Reinforcement Learning
2025 · Nicolas Castanet, Olivier Sigaud, Sylvain Lamprier
Abstract
Goal-Conditioned Reinforcement Learning (GCRL) enables agents to autonomously acquire diverse behaviors, but faces major challenges in visual environments due to high-dimensional, semantically sparse observations. In the online setting, where agents learn representations while exploring, the latent space evolves with the agent's policy, to capture newly discovered areas of the environment. However, without incentivization to maximize state coverage in the representation, classical approaches based on auto-encoders may converge to latent spaces that over-represent a restricted set of states frequently visited by the agent. This is exacerbated in an intrinsic motivation setting, where the agent uses the distribution encoded in the latent space to sample the goals it learns to master. To address this issue, we propose to progressively enforce distributional shifts towards a uniform distribution over the full state space, to ensure a full coverage of skills that can be learned in the envir
Authors
(none)
Tags
Stats
Related papers
- Dense And Diverse Goal Coverage In Multi Goal Reinforcement Learning (2025)0.00
- Goal-conditioned Offline Reinforcement Learning Through State Space Partitioning (2023)2.26
- Distributionally Robust Model-based Reinforcement Learning With Large State Spaces (2023)0.00
- What Can Online Reinforcement Learning With Function Approximation Benefit From General Coverage Conditions? (2023)0.00
- Distributionally Robust Online Markov Game With Linear Function Approximation (2025)0.00
- Learning To Explore When Mistakes Are Not Allowed (2025)0.00
- Provably Efficient Offline Goal-conditioned Reinforcement Learning With General Function Approximation And Single-policy Concentrability (2023)0.00
- Decision Mamba: A Multi-grained State Space Model With Self-evolution Regularization For Offline RL (2024)0.00