Learning Dynamic Abstract Representations For Sample-efficient Reinforcement Learning

Abstract

In many real-world problems, the learning agent needs to learn a problem's abstractions and solution simultaneously. However, most such abstractions need to be designed and refined by hand for different problems and domains of application. This paper presents a novel top-down approach for constructing state abstractions while carrying out reinforcement learning. Starting with state variables and a simulator, it presents a novel domain-independent approach for dynamically computing an abstraction based on the dispersion of Q-values in abstract states as the agent continues acting and learning. Extensive empirical evaluation on multiple domains and problems shows that this approach automatically learns abstractions that are finely-tuned to the problem, yield powerful sample efficiency, and result in the RL agent significantly outperforming existing approaches.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keydadvar2022learning

Related papers

• A Theory Of Abstraction In Reinforcement Learning (2022)0.00
• Localized Observation Abstraction Using Piecewise Linear Spatial Decay For Reinforcement Learning In Combat Simulations (2024)0.00
• Contrastive Abstraction For Reinforcement Learning (2024)0.00
• Learning Markov State Abstractions For Deep Reinforcement Learning (2021)0.00
• Unsupervised Representation Learning In Deep Reinforcement Learning: A Review (2022)9.59
• Discrete State-action Abstraction Via The Successor Representation (2022)0.00
• Leveraging Class Abstraction For Commonsense Reinforcement Learning Via Residual Policy Gradient Methods (2022)0.00
• Learning State Abstractions For Transfer In Continuous Control (2020)0.00