Epistemic Risk-sensitive Reinforcement Learning

Abstract

We develop a framework for interacting with uncertain environments in reinforcement learning (RL) by leveraging preferences in the form of utility functions. We claim that there is value in considering different risk measures during learning. In this framework, the preference for risk can be tuned by variation of the parameter \(\beta\) and the resulting behavior can be risk-averse, risk-neutral or risk-taking depending on the parameter choice. We evaluate our framework for learning problems with model uncertainty. We measure and control for *epistemic* risk using dynamic programming (DP) and policy gradient-based algorithms. The risk-averse behavior is then compared with the behavior of the optimal risk-neutral policy in environments with epistemic risk.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyeriksson2019epistemic

Related papers

• Robust Bayesian Dynamic Programming For On-policy Risk-sensitive Reinforcement Learning (2025)0.00
• Ra-pbrl: Provably Efficient Risk-aware Preference-based Reinforcement Learning (2024)0.00
• Online Bayesian Risk-averse Reinforcement Learning (2025)0.00
• Conditionally Elicitable Dynamic Risk Measures For Deep Reinforcement Learning (2022)0.00
• Ergodic Risk Measures: Towards A Risk-aware Foundation For Continual Reinforcement Learning (2025)0.00
• Optimism As Risk-seeking In Multi-agent Reinforcement Learning (2025)0.00
• Risk-sensitive Bayesian Games For Multi-agent Reinforcement Learning Under Policy Uncertainty (2022)0.00
• A Risk-sensitive Approach To Policy Optimization (2022)3.58