Ergodic Risk Measures: Towards A Risk-aware Foundation For Continual Reinforcement Learning

Abstract

Continual reinforcement learning (continual RL) seeks to formalize the notions of lifelong learning and endless adaptation in RL. In particular, the aim of continual RL is to develop RL agents that can maintain a careful balance between retaining useful information and adapting to new situations. To date, continual RL has been explored almost exclusively through the lens of risk-neutral decision-making, in which the agent aims to optimize the expected long-run performance. In this work, we present the first formal theoretical treatment of continual RL through the lens of risk-aware decision-making, in which the behaviour of the agent is directed towards optimizing a measure of long-run performance beyond the mean. In particular, we show that the classical theory of risk measures, widely used as a theoretical foundation in non-continual risk-aware RL, is, in its current form, incompatible with continual learning. Then, building on this insight, we extend risk measure theory into the con

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyrojas2025ergodic

Related papers

• Conditionally Elicitable Dynamic Risk Measures For Deep Reinforcement Learning (2022)0.00
• Towards Continual Reinforcement Learning: A Review And Perspectives (2020)0.00
• Safe Continual Reinforcement Learning In Non-stationary Environments (2026)12.89
• Epistemic Risk-sensitive Reinforcement Learning (2019)0.00
• One Risk To Rule Them All: A Risk-sensitive Perspective On Model-based Offline Reinforcement Learning (2022)3.58
• Extreme Risk Mitigation In Reinforcement Learning Using Extreme Value Theory (2023)0.00
• DRL-ORA: Distributional Reinforcement Learning With Online Risk Adaption (2023)0.00
• Online Bayesian Risk-averse Reinforcement Learning (2025)0.00