Joint Optimization Of Multi-objective Reinforcement Learning With Policy Gradient Based Algorithm

Abstract

Many engineering problems have multiple objectives, and the overall aim is to optimize a non-linear function of these objectives. In this paper, we formulate the problem of maximizing a non-linear concave function of multiple long-term objectives. A policy-gradient based model-free algorithm is proposed for the problem. To compute an estimate of the gradient, a biased estimator is proposed. The proposed algorithm is shown to achieve convergence to within an \(\epsilon\) of the global optima after sampling \(\mathcal\{O\}(\frac\{M^4\sigma^2\}\{(1-\gamma)^8\epsilon^4\})\) trajectories where \(\gamma\) is the discount factor and \(M\) is the number of the agents, thus achieving the same dependence on \(\epsilon\) as the policy gradient algorithm for the standard reinforcement learning.

Stats

• citations6
• S2 citations—
• github stars0
• HF likes0
• heat score6.34
• arxiv keybai2021joint

Related papers

• Breaking The Bias Barrier In Concave Multi-objective Reinforcement Learning (2026)0.00
• Anchor-changing Regularized Natural Policy Gradient For Multi-objective Reinforcement Learning (2022)0.00
• Sample-efficient Multi-objective Learning Via Generalized Policy Improvement Prioritization (2023)5.24
• Natural Policy Gradient And Actor Critic Methods For Constrained Multi-task Reinforcement Learning (2024)0.00
• A Generalized Algorithm For Multi-objective Reinforcement Learning And Policy Adaptation (2019)0.00
• Local Optimization Achieves Global Optimality In Multi-agent Reinforcement Learning (2023)0.00
• Global Convergence Of Policy Gradient Methods In Reinforcement Learning, Games And Control (2023)0.00
• A Policy Gradient Algorithm For Learning To Learn In Multiagent Reinforcement Learning (2020)0.00