DRL-ORA: Distributional Reinforcement Learning With Online Risk Adaption

Abstract

One of the main challenges in reinforcement learning (RL) is that the agent has to make decisions that would influence the future performance without having complete knowledge of the environment. Dynamically adjusting the level of epistemic risk during the learning process can help to achieve reliable policies in safety-critical settings with better efficiency. In this work, we propose a new framework, Distributional RL with Online Risk Adaptation (DRL-ORA). This framework quantifies both epistemic and implicit aleatory uncertainties in a unified manner and dynamically adjusts the epistemic risk levels by solving a total variation minimization problem online. The framework unifies the existing variants of risk adaption approaches and offers better explainability and flexibility. The selection of risk levels is performed efficiently via a grid search using a Follow-The-Leader-type algorithm, where the offline oracle also corresponds to a ''satisficing measure'' under a specially modifie

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywu2023drl

Related papers

• One Risk To Rule Them All: A Risk-sensitive Perspective On Model-based Offline Reinforcement Learning (2022)3.58
• Online Bayesian Risk-averse Reinforcement Learning (2025)0.00
• A Risk-sensitive Approach To Policy Optimization (2022)3.58
• Sample-efficient Distributionally Robust Multi-agent Reinforcement Learning Via Online Interaction (2025)0.00
• On The Foundation Of Distributionally Robust Reinforcement Learning (2023)0.00
• Bridging Distributional And Risk-sensitive Reinforcement Learning With Provable Regret Bounds (2022)0.00
• Bridging Distributionally Robust Learning And Offline RL: An Approach To Mitigate Distribution Shift And Partial Data Coverage (2023)0.00
• Ergodic Risk Measures: Towards A Risk-aware Foundation For Continual Reinforcement Learning (2025)0.00