Demystifying Reinforcement Learning In Time-varying Systems

Abstract

Recent research has turned to Reinforcement Learning (RL) to solve challenging decision problems, as an alternative to hand-tuned heuristics. RL can learn good policies without the need for modeling the environment's dynamics. Despite this promise, RL remains an impractical solution for many real-world systems problems. A particularly challenging case occurs when the environment changes over time, i.e. it exhibits non-stationarity. In this work, we characterize the challenges introduced by non-stationarity, shed light on the range of approaches to them and develop a robust framework for addressing them to train RL agents in live systems. Such agents must explore and learn new environments, without hurting the system's performance, and remember them over time. To this end, our framework (i) identifies different environments encountered by the live system, (ii) triggers exploration when necessary, (iii) takes precautions to retain knowledge from prior environments, and (iv) employs safeg

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyhamadanian2022demystifying

Related papers

• Model-agnostic Solutions For Deep Reinforcement Learning In Non-ergodic Contexts (2026)0.00
• Safe Continual Reinforcement Learning In Non-stationary Environments (2026)12.89
• Tempo Adaptation In Non-stationary Reinforcement Learning (2023)0.00
• Online Reinforcement Learning In Non-stationary Context-driven Environments (2023)0.00
• Reinforcement Learning For Control Systems With Time Delays: A Comprehensive Survey (2026)0.00
• Explaining Online Reinforcement Learning Decisions Of Self-adaptive Systems (2022)9.59
• Reinforcement Learning With Non-ergodic Reward Increments: Robustness Via Ergodicity Transformations (2023)0.00
• Reinforcement Learning With Fast Stabilization In Linear Dynamical Systems (2020)0.00