On Learning Intrinsic Rewards For Policy Gradient Methods

Abstract

In many sequential decision making tasks, it is challenging to design reward functions that help an RL agent efficiently learn behavior that is considered good by the agent designer. A number of different formulations of the reward-design problem, or close variants thereof, have been proposed in the literature. In this paper we build on the Optimal Rewards Framework of Singh et.al. that defines the optimal intrinsic reward function as one that when used by an RL agent achieves behavior that optimizes the task-specifying or extrinsic reward function. Previous work in this framework has shown how good intrinsic reward functions can be learned for lookahead search based planning agents. Whether it is possible to learn intrinsic reward functions for learning agents remains an open problem. In this paper we derive a novel algorithm for learning intrinsic rewards for policy-gradient based learning agents. We compare the performance of an augmented agent that uses our algorithm to provide add

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzheng2018on

Related papers

• What Can Learned Intrinsic Rewards Capture? (2019)0.00
• Intrinsic Reward Policy Optimization For Sparse-reward Environments (2026)0.00
• Redeeming Intrinsic Rewards Via Constrained Optimization (2022)0.00
• Improving Policy Gradient By Exploring Under-appreciated Rewards (2016)0.00
• Reward Design For Reinforcement Learning Agents (2025)0.00
• Rlexplore: Accelerating Research In Intrinsically-motivated Reinforcement Learning (2024)5.33
• Coordinated Exploration Via Intrinsic Rewards For Multi-agent Reinforcement Learning (2019)0.00
• Model-free Policy Learning With Reward Gradients (2021)0.00