Learning State Abstractions For Transfer In Continuous Control

Abstract

Can simple algorithms with a good representation solve challenging reinforcement learning problems? In this work, we answer this question in the affirmative, where we take "simple learning algorithm" to be tabular Q-Learning, the "good representations" to be a learned state abstraction, and "challenging problems" to be continuous control tasks. Our main contribution is a learning algorithm that abstracts a continuous state-space into a discrete one. We transfer this learned representation to unseen problems to enable effective learning. We provide theory showing that learned abstractions maintain a bounded value loss, and we report experiments showing that the abstractions empower tabular Q-Learning to learn efficiently in unseen tasks.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyasadi2020learning

Related papers

• How To Discretize Continuous State-action Spaces In Q-learning: A Symbolic Control Approach (2024)3.58
• Learning Robust And Adaptive Real-world Continuous Control Using Simulation And Transfer Learning (2018)0.00
• Discrete State-action Abstraction Via The Successor Representation (2022)0.00
• Learning Dynamic Abstract Representations For Sample-efficient Reinforcement Learning (2022)0.00
• Learning Markov State Abstractions For Deep Reinforcement Learning (2021)0.00
• Specialized Deep Residual Policy Safe Reinforcement Learning-based Controller For Complex And Continuous State-action Spaces (2023)4.52
• On The Geometry Of Reinforcement Learning In Continuous State And Action Spaces (2022)0.00
• A Tour Of Reinforcement Learning: The View From Continuous Control (2018)19.86