Policy Information Capacity: Information-theoretic Measure For Task Complexity In Deep Reinforcement Learning

Abstract

Progress in deep reinforcement learning (RL) research is largely enabled by benchmark task environments. However, analyzing the nature of those environments is often overlooked. In particular, we still do not have agreeable ways to measure the difficulty or solvability of a task, given that each has fundamentally different actions, observations, dynamics, rewards, and can be tackled with diverse RL algorithms. In this work, we propose policy information capacity (PIC) -- the mutual information between policy parameters and episodic return -- and policy-optimal information capacity (POIC) -- between policy parameters and episodic optimality -- as two environment-agnostic, algorithm-agnostic quantitative metrics for task difficulty. Evaluating our metrics across toy environments as well as continuous control benchmark tasks from OpenAI Gym and DeepMind Control Suite, we empirically demonstrate that these information-theoretic metrics have higher correlations with normalized task solvabil

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyfuruta2021policy

Related papers

• Mutual Information Tracks Policy Coherence In Reinforcement Learning (2025)0.00
• Deep RL With Information Constrained Policies: Generalization In Continuous Control (2020)0.00
• Inforl: Interpretable Reinforcement Learning Using Information Maximization (2019)0.00
• Rethinking Model-based, Policy-based, And Value-based Reinforcement Learning Via The Lens Of Representation Complexity (2023)2.26
• Addressing Action Oscillations Through Learning Policy Inertia (2021)7.81
• Occupancy Information Ratio: Infinite-horizon, Information-directed, Parameterized Policy Search (2022)0.00
• What Matters In On-policy Reinforcement Learning? A Large-scale Empirical Study (2020)0.00
• Consolidation Via Policy Information Regularization In Deep RL For Multi-agent Games (2020)0.00