Internally Rewarded Reinforcement Learning

Abstract

We study a class of reinforcement learning problems where the reward signals for policy learning are generated by an internal reward model that is dependent on and jointly optimized with the policy. This interdependence between the policy and the reward model leads to an unstable learning process because reward signals from an immature reward model are noisy and impede policy learning, and conversely, an under-optimized policy impedes reward estimation learning. We call this learning setting \(\textit\{Internally Rewarded Reinforcement Learning\}\) (IRRL) as the reward is not provided directly by the environment but \(\textit\{internally\}\) by a reward model. In this paper, we formally formulate IRRL and present a class of problems that belong to IRRL. We theoretically derive and empirically analyze the effect of the reward function in IRRL and based on these analyses propose the clipped linear reward function. Experimental results show that the proposed reward function can consistent

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyli2023internally

Related papers

• Maximum-likelihood Inverse Reinforcement Learning With Finite-time Guarantees (2022)0.00
• Towards Theoretical Understanding Of Inverse Reinforcement Learning (2023)0.00
• Inverse Reinforcement Learning Using Just Classification And A Few Regressions (2025)0.00
• Inverse Reinforcement Learning With Simultaneous Estimation Of Rewards And Dynamics (2016)0.00
• A Survey Of Inverse Reinforcement Learning: Challenges, Methods And Progress (2018)0.00
• Policy Improvement Reinforcement Learning (2026)0.00
• Information Directed Reward Learning For Reinforcement Learning (2021)0.00
• Intrinsic Reward Policy Optimization For Sparse-reward Environments (2026)0.00