Experimental Evidence That Empowerment May Drive Exploration In Sparse-reward Environments

Abstract

Reinforcement Learning (RL) is known to be often unsuccessful in environments with sparse extrinsic rewards. A possible countermeasure is to endow RL agents with an intrinsic reward function, or 'intrinsic motivation', which rewards the agent based on certain features of the current sensor state. An intrinsic reward function based on the principle of empowerment assigns rewards proportional to the amount of control the agent has over its own sensors. We implemented a variation on a recently proposed intrinsically motivated agent, which we refer to as the 'curious' agent, and an empowerment-inspired agent. The former leverages sensor state encoding with a variational autoencoder, while the latter predicts the next sensor state via a variational information bottleneck. We compared the performance of both agents to that of an advantage actor-critic baseline in four sparse reward grid worlds. Both the empowerment agent and its curious competitor seem to benefit to similar extents from thei

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymassari2021experimental

Related papers

• A Unified Strategy For Implementing Curiosity And Empowerment Driven Reinforcement Learning (2018)0.00
• Intrinsic Rewards For Exploration Without Harm From Observational Noise: A Simulation Study Based On The Free Energy Principle (2024)0.00
• A Unified Bellman Optimality Principle Combining Reward Maximization And Empowerment (2019)0.00
• The Impact Of Intrinsic Rewards On Exploration In Reinforcement Learning (2025)0.00
• Redeeming Intrinsic Rewards Via Constrained Optimization (2022)0.00
• Rlexplore: Accelerating Research In Intrinsically-motivated Reinforcement Learning (2024)5.33
• Curiosity-driven Exploration In Sparse-reward Multi-agent Reinforcement Learning (2023)0.00
• Information Content Exploration (2023)0.00