Online Reinforcement Learning With Sparse Rewards Through An Active Inference Capsule

Abstract

Intelligent agents must pursue their goals in complex environments with partial information and often limited computational capacity. Reinforcement learning methods have achieved great success by creating agents that optimize engineered reward functions, but which often struggle to learn in sparse-reward environments, generally require many environmental interactions to perform well, and are typically computationally very expensive. Active inference is a model-based approach that directs agents to explore uncertain states while adhering to a prior model of their goal behaviour. This paper introduces an active inference agent which minimizes the novel free energy of the expected future. Our model is capable of solving sparse-reward problems with a very high sample efficiency due to its objective function, which encourages directed exploration of uncertain states. Moreover, our model is computationally very light and can operate in a fully online manner while achieving comparable perform

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keynoel2021online

Related papers

• Reward Maximisation Through Discrete Active Inference (2020)10.74
• Prior Preference Learning From Experts:designing A Reward With Active Inference (2021)0.00
• R-AIF: Solving Sparse-reward Robotic Tasks From Pixels With Active Inference And World Models (2024)4.52
• Deep Active Inference Agents Using Monte-carlo Methods (2020)0.00
• Coordinated Exploration Via Intrinsic Rewards For Multi-agent Reinforcement Learning (2019)0.00
• Active Inference And Reinforcement Learning: A Unified Inference On Continuous State And Action Spaces Under Partial Observability (2022)5.84
• Long-term Visitation Value For Deep Exploration In Sparse Reward Reinforcement Learning (2020)7.24
• Information Content Exploration (2023)0.00