Active Inference And Reinforcement Learning: A Unified Inference On Continuous State And Action Spaces Under Partial Observability

Abstract

Reinforcement learning (RL) has garnered significant attention for developing decision-making agents that aim to maximize rewards, specified by an external supervisor, within fully observable environments. However, many real-world problems involve partial observations, formulated as partially observable Markov decision processes (POMDPs). Previous studies have tackled RL in POMDPs by either incorporating the memory of past actions and observations or by inferring the true state of the environment from observed data. However, aggregating observed data over time becomes impractical in continuous spaces. Moreover, inference-based RL approaches often require many samples to perform well, as they focus solely on reward maximization and neglect uncertainty in the inferred state. Active inference (AIF) is a framework formulated in POMDPs and directs agents to select actions by minimizing a function called expected free energy (EFE). This supplies reward-maximizing (exploitative) behaviour, as

Stats

• citations5
• S2 citations—
• github stars0
• HF likes0
• heat score5.84
• arxiv keymalekzadeh2022active

Related papers

• Active Inference: Demystified And Compared (2019)15.98
• R-AIF: Solving Sparse-reward Robotic Tasks From Pixels With Active Inference And World Models (2024)4.52
• Reinforcement Learning Under Partial Observability Guided By Learned Environment Models (2022)6.34
• Optimal Decision-making In Mixed-agent Partially Observable Stochastic Environments Via Reinforcement Learning (2019)0.00
• Goal-oriented Inference Of Environment From Redundant Observations (2023)3.58
• Act-then-measure: Reinforcement Learning For Partially Observable Environments With Active Measuring (2023)3.58
• Reward Maximisation Through Discrete Active Inference (2020)10.74
• Deep Active Inference For Partially Observable Mdps (2020)9.59