A Theory Of Abstraction In Reinforcement Learning

Abstract

Reinforcement learning defines the problem facing agents that learn to make good decisions through action and observation alone. To be effective problem solvers, such agents must efficiently explore vast worlds, assign credit from delayed feedback, and generalize to new experiences, all while making use of limited data, computational resources, and perceptual bandwidth. Abstraction is essential to all of these endeavors. Through abstraction, agents can form concise models of their environment that support the many practices required of a rational, adaptive decision maker. In this dissertation, I present a theory of abstraction in reinforcement learning. I first offer three desiderata for functions that carry out the process of abstraction: they should 1) preserve representation of near-optimal behavior, 2) be learned and constructed efficiently, and 3) lower planning or learning time. I then present a suite of new algorithms and analysis that clarify how agents can learn to abstract ac

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyabel2022a

Related papers

• Learning Dynamic Abstract Representations For Sample-efficient Reinforcement Learning (2022)0.00
• Autonomous Option Invention For Continual Hierarchical Reinforcement Learning And Planning (2024)2.26
• Contrastive Abstraction For Reinforcement Learning (2024)0.00
• Discrete State-action Abstraction Via The Successor Representation (2022)0.00
• Environment Shaping In Reinforcement Learning Using State Abstraction (2020)0.00
• Human-level Reinforcement Learning Through Theory-based Modeling, Exploration, And Planning (2021)0.00
• An Analysis Of Model-based Reinforcement Learning From Abstracted Observations (2022)0.00
• Learning Markov State Abstractions For Deep Reinforcement Learning (2021)0.00