Reward Maximisation Through Discrete Active Inference

Abstract

Active inference is a probabilistic framework for modelling the behaviour of biological and artificial agents, which derives from the principle of minimising free energy. In recent years, this framework has successfully been applied to a variety of situations where the goal was to maximise reward, offering comparable and sometimes superior performance to alternative approaches. In this paper, we clarify the connection between reward maximisation and active inference by demonstrating how and when active inference agents perform actions that are optimal for maximising reward. Precisely, we show the conditions under which active inference produces the optimal solution to the Bellman equation--a formulation that underlies several approaches to model-based reinforcement learning and control. On partially observed Markov decision processes, the standard active inference scheme can produce Bellman optimal actions for planning horizons of 1, but not beyond. In contrast, a recently developed re

Stats

• citations26
• S2 citations—
• github stars0
• HF likes0
• heat score10.74
• arxiv keydacosta2020reward

Related papers

• Active Inference As A Model Of Agency (2024)0.00
• Deep Active Inference Agents Using Monte-carlo Methods (2020)0.00
• Deconstructing Deep Active Inference (2023)2.26
• Active Inference: Demystified And Compared (2019)15.98
• Sophisticated Inference (2020)14.83
• Prior Preference Learning From Experts:designing A Reward With Active Inference (2021)0.00
• Online Reinforcement Learning With Sparse Rewards Through An Active Inference Capsule (2021)0.00
• Active Inference And Reinforcement Learning: A Unified Inference On Continuous State And Action Spaces Under Partial Observability (2022)5.84