Optimal Control With Natural Images: Efficient Reinforcement Learning Using Overcomplete Sparse Codes

Abstract

Optimal control and sequential decision making are widely used in many complex tasks. Optimal control over a sequence of natural images is a first step towards understanding the role of vision in control. Here, we formalize this problem as a reinforcement learning task, and derive general conditions under which an image includes enough information to implement an optimal policy. Reinforcement learning is shown to provide a computationally efficient method for finding optimal policies when natural images are encoded into "efficient" image representations. This is demonstrated by introducing a new reinforcement learning benchmark that easily scales to large numbers of states and long horizons. In particular, by representing each image as an overcomplete sparse code, we are able to efficiently solve an optimal control task that is orders of magnitude larger than those tasks solvable using complete codes. Theoretical justification for this behaviour is provided. This work also demonstrates

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyloxley2024optimal

Related papers

• Improving Sample Efficiency In Model-free Reinforcement Learning From Images (2019)16.99
• The Utility Of Sparse Representations For Control In Reinforcement Learning (2018)10.48
• RLOC: Neurobiologically Inspired Hierarchical Reinforcement Learning Algorithm For Continuous Control Of Nonlinear Dynamical Systems (2019)0.00
• From Reinforcement Learning To Optimal Control: A Unified Framework For Sequential Decisions (2019)0.00
• Actively Learning Reinforcement Learning: A Stochastic Optimal Control Approach (2023)0.00
• An Optical Control Environment For Benchmarking Reinforcement Learning Algorithms (2022)0.00
• Playing Atari With Six Neurons (2018)0.00
• Learning Sparse Representations In Reinforcement Learning With Sparse Coding (2017)8.82