Modulation Of Viability Signals For Self-regulatory Control

Abstract

We revisit the role of instrumental value as a driver of adaptive behavior. In active inference, instrumental or extrinsic value is quantified by the information-theoretic surprisal of a set of observations measuring the extent to which those observations conform to prior beliefs or preferences. That is, an agent is expected to seek the type of evidence that is consistent with its own model of the world. For reinforcement learning tasks, the distribution of preferences replaces the notion of reward. We explore a scenario in which the agent learns this distribution in a self-supervised manner. In particular, we highlight the distinction between observations induced by the environment and those pertaining more directly to the continuity of an agent in time. We evaluate our methodology in a dynamic environment with discrete time and actions. First with a surprisal minimizing model-free agent (in the RL sense) and then expanding to the model-based case to minimize the expected free energy.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyovalle2020modulation

Related papers

• Is There Value In Reinforcement Learning? (2025)0.00
• On The Limited Representational Power Of Value Functions And Its Links To Statistical (in)efficiency (2024)0.00
• Exploration And Preference Satisfaction Trade-off In Reward-free Learning (2021)0.00
• Information Is Power: Intrinsic Control Via Information Capture (2021)0.00
• Surprise-adaptive Intrinsic Motivation For Unsupervised Reinforcement Learning (2024)0.00
• Variational Inference For Model-free And Model-based Reinforcement Learning (2022)0.00
• Deciding What To Model: Value-equivalent Sampling For Reinforcement Learning (2022)0.00
• Active Inference As A Model Of Agency (2024)0.00