Learning General Parameterized Policies For Infinite Horizon Average Reward Constrained Mdps Via Primal-dual Policy Gradient Algorithm

Abstract

This paper explores the realm of infinite horizon average reward Constrained Markov Decision Processes (CMDPs). To the best of our knowledge, this work is the first to delve into the regret and constraint violation analysis of average reward CMDPs with a general policy parametrization. To address this challenge, we propose a primal dual-based policy gradient algorithm that adeptly manages the constraints while ensuring a low regret guarantee toward achieving a global optimal policy. In particular, our proposed algorithm achieves \(\tilde\{\mathcal\{O\}\}(\{T\}^\{4/5\})\) objective regret and \(\tilde\{\mathcal\{O\}\}(\{T\}^\{4/5\})\) constraint violation bounds.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keybai2024learning

Related papers

• A Policy Gradient Primal-dual Algorithm For Constrained Mdps With Uniform PAC Guarantees (2024)0.00
• Last-iterate Convergence Of General Parameterized Policies In Constrained Mdps (2026)0.00
• Policy Optimization For Constrained Mdps With Provable Fast Global Convergence (2021)0.00
• Reinforcement Learning For Infinite-horizon Average-reward Linear Mdps Via Approximation By Discounted-reward Mdps (2024)0.00
• A Policy Gradient Approach For Finite Horizon Constrained Markov Decision Processes (2022)3.58
• Stochastic First-order Methods For Average-reward Markov Decision Processes (2022)3.58
• Provably Efficient Primal-dual Reinforcement Learning For Cmdps With Non-stationary Objectives And Constraints (2022)0.00
• Efficient Policy Optimization In Robust Constrained Mdps With Iteration Complexity Guarantees (2025)0.00