Risk-sensitive Reinforcement Learning With Exponential Criteria

Abstract

While reinforcement learning has shown experimental success in a number of applications, it is known to be sensitive to noise and perturbations in the parameters of the system, leading to high variance in the total reward amongst different episodes in slightly different environments. To introduce robustness, as well as sample efficiency, risk-sensitive reinforcement learning methods are being thoroughly studied. In this work, we provide a definition of robust reinforcement learning policies and formulate a risk-sensitive reinforcement learning problem to approximate them, by solving an optimization problem with respect to a modified objective based on exponential criteria. In particular, we study a model-free risk-sensitive variation of the widely-used Monte Carlo Policy Gradient algorithm and introduce a novel risk-sensitive online Actor-Critic algorithm based on solving a multiplicative Bellman equation using stochastic approximation updates. Analytical results suggest that the use o

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keynoorani2022risk

Related papers

• Robust Bayesian Dynamic Programming For On-policy Risk-sensitive Reinforcement Learning (2025)0.00
• Exponential Bellman Equation And Improved Regret Bounds For Risk-sensitive Reinforcement Learning (2021)0.00
• Bounded Robustness In Reinforcement Learning Via Lexicographic Objectives (2022)0.00
• Continuous-time Risk-sensitive Reinforcement Learning Via Quadratic Variation Penalty (2024)0.00
• A Risk-sensitive Approach To Policy Optimization (2022)3.58
• A Policy Gradient Approach For Optimization Of Smooth Risk Measures (2022)0.00
• Extreme Risk Mitigation In Reinforcement Learning Using Extreme Value Theory (2023)0.00
• Model-based Epistemic Variance Of Values For Risk-aware Policy Optimization (2023)0.00