Reinforcement Learning With Non-cumulative Objective

Abstract

In reinforcement learning, the objective is almost always defined as a *cumulative* function over the rewards along the process. However, there are many optimal control and reinforcement learning problems in various application fields, especially in communications and networking, where the objectives are not naturally expressed as summations of the rewards. In this paper, we recognize the prevalence of non-cumulative objectives in various problems, and propose a modification to existing algorithms for optimizing such objectives. Specifically, we dive into the fundamental building block for many optimal control and reinforcement learning algorithms: the Bellman optimality equation. To optimize a non-cumulative objective, we replace the original summation operation in the Bellman update rule with a generalized operation corresponding to the objective. Furthermore, we provide sufficient conditions on the form of the generalized operation as well as assumptions on the Markov decision proce

Stats

• citations4
• S2 citations—
• github stars0
• HF likes0
• heat score5.24
• arxiv keycui2023reinforcement

Related papers

• Computably Continuous Reinforcement-learning Objectives Are Pac-learnable (2023)7.81
• Average Reward Reinforcement Learning For Omega-regular And Mean-payoff Objectives (2025)0.00
• Analyzing And Bridging The Gap Between Maximizing Total Reward And Discounted Reward In Deep Reinforcement Learning (2024)0.00
• Global Reinforcement Learning: Beyond Linear And Convex Rewards Via Submodular Semi-gradient Methods (2024)0.00
• Joint Optimization Of Multi-objective Reinforcement Learning With Policy Gradient Based Algorithm (2021)6.34
• Examining Average And Discounted Reward Optimality Criteria In Reinforcement Learning (2021)0.00
• Breaking The Bias Barrier In Concave Multi-objective Reinforcement Learning (2026)0.00
• A Kernel Loss For Solving The Bellman Equation (2019)0.00