Generative Intrinsic Optimization: Intrinsic Control With Model Learning

Abstract

Future sequence represents the outcome after executing the action into the environment (i.e. the trajectory onwards). When driven by the information-theoretic concept of mutual information, it seeks maximally informative consequences. Explicit outcomes may vary across state, return, or trajectory serving different purposes such as credit assignment or imitation learning. However, the inherent nature of incorporating intrinsic motivation with reward maximization is often neglected. In this work, we propose a policy iteration scheme that seamlessly incorporates the mutual information, ensuring convergence to the optimal policy. Concurrently, a variational approach is introduced, which jointly learns the necessary quantity for estimating the mutual information and the dynamics model, providing a general framework for incorporating different forms of outcomes of interest. While we mainly focus on theoretical analysis, our approach opens the possibilities of leveraging intrinsic control wit

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyma2023generative

Related papers

• On Learning Intrinsic Rewards For Policy Gradient Methods (2018)0.00
• Deep Intrinsically Motivated Exploration In Continuous Control (2022)0.00
• Information Is Power: Intrinsic Control Via Information Capture (2021)0.00
• Variational Intrinsic Control Revisited (2020)0.00
• Generative Models In Decision Making: A Survey (2025)0.00
• Counterfactual Control For Free From Generative Models (2017)0.00
• Scheduled Intrinsic Drive: A Hierarchical Take On Intrinsically Motivated Exploration (2019)0.00
• What Can Learned Intrinsic Rewards Capture? (2019)0.00