Information Content Exploration

Abstract

Sparse reward environments are known to be challenging for reinforcement learning agents. In such environments, efficient and scalable exploration is crucial. Exploration is a means by which an agent gains information about the environment. We expand on this topic and propose a new intrinsic reward that systemically quantifies exploratory behavior and promotes state coverage by maximizing the information content of a trajectory taken by an agent. We compare our method to alternative exploration based intrinsic reward techniques, namely Curiosity Driven Learning and Random Network Distillation. We show that our information theoretic reward induces efficient exploration and outperforms in various games, including Montezuma Revenge, a known difficult task for reinforcement learning. Finally, we propose an extension that maximizes information content in a discretely compressed latent space which boosts sample efficiency and generalizes to continuous state spaces.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keychmura2023information

Related papers

• Curiosity-driven Multi-agent Exploration With Mixed Objectives (2022)0.00
• Curiosity-driven Exploration In Sparse-reward Multi-agent Reinforcement Learning (2023)0.00
• Intrinsic Reward Policy Optimization For Sparse-reward Environments (2026)0.00
• The Impact Of Intrinsic Rewards On Exploration In Reinforcement Learning (2025)0.00
• Intrinsic Rewards For Exploration Without Harm From Observational Noise: A Simulation Study Based On The Free Energy Principle (2024)0.00
• R\'enyi State Entropy For Exploration Acceleration In Reinforcement Learning (2022)0.00
• Long-term Visitation Value For Deep Exploration In Sparse Reward Reinforcement Learning (2020)7.24
• Self-supervised Exploration Via Temporal Inconsistency In Reinforcement Learning (2022)3.58