On The Foundation Of Distributionally Robust Reinforcement Learning

Abstract

Motivated by the need for a robust policy in the face of environment shifts between training and deployment, we contribute to the theoretical foundation of distributionally robust reinforcement learning (DRRL). This is accomplished through a comprehensive modeling framework centered around robust Markov decision processes (RMDPs). This framework obliges the decision maker to choose an optimal policy under the worst-case distributional shift orchestrated by an adversary. By unifying and extending existing formulations, we rigorously construct RMDPs that embrace various modeling attributes for both the decision maker and the adversary. These attributes include the structure of information availability-covering history-dependent, Markov, and Markov time-homogeneous dynamics-as well as constraints on the shifts induced by the adversary, with a focus on SA- and S-rectangularity. Within this RMDP framework, we investigate conditions for the existence or absence of the dynamic programming pri

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywang2023on

Related papers

• The Curious Price Of Distributional Robustness In Reinforcement Learning With A Generative Model (2023)0.00
• Distributionally Robust Self Paced Curriculum Reinforcement Learning (2025)0.00
• Single-trajectory Distributionally Robust Reinforcement Learning (2023)0.00
• Improving Robustness Via Risk Averse Distributional Reinforcement Learning (2020)0.00
• Linear Mixture Distributionally Robust Markov Decision Processes (2025)0.00
• Sample-efficient Robust Multi-agent Reinforcement Learning In The Face Of Environmental Uncertainty (2024)0.00
• Robust Bayesian Dynamic Programming For On-policy Risk-sensitive Reinforcement Learning (2025)0.00
• Adversary Agnostic Robust Deep Reinforcement Learning (2020)6.77