Deep RL With Information Constrained Policies: Generalization In Continuous Control

Abstract

Biological agents learn and act intelligently in spite of a highly limited capacity to process and store information. Many real-world problems involve continuous control, which represents a difficult task for artificial intelligence agents. In this paper we explore the potential learning advantages a natural constraint on information flow might confer onto artificial agents in continuous control tasks. We focus on the model-free reinforcement learning (RL) setting and formalize our approach in terms of an information-theoretic constraint on the complexity of learned policies. We show that our approach emerges in a principled fashion from the application of rate-distortion theory. We implement a novel Capacity-Limited Actor-Critic (CLAC) algorithm and situate it within a broader family of RL algorithms such as the Soft Actor Critic (SAC) and Mutual Information Reinforcement Learning (MIRL) algorithm. Our experiments using continuous control tasks show that compared to alternative approa

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymalloy2020deep

Related papers

• Attraction-repulsion Actor-critic For Continuous Control Reinforcement Learning (2019)0.00
• Consolidation Via Policy Information Regularization In Deep RL For Multi-agent Games (2020)0.00
• Discrete And Continuous Action Representation For Practical RL In Video Games (2019)0.00
• Action-adaptive Continual Learning: Enabling Policy Generalization Under Dynamic Action Spaces (2025)0.00
• Continuous Control With Contexts, Provably (2019)0.00
• Broad Critic Deep Actor Reinforcement Learning For Continuous Control (2024)0.00
• Dynamics Generalization Via Information Bottleneck In Deep Reinforcement Learning (2020)0.00
• Deep Exploration With Pac-bayes (2024)0.00