Robust Anytime Learning Of Markov Decision Processes

Abstract

Markov decision processes (MDPs) are formal models commonly used in sequential decision-making. MDPs capture the stochasticity that may arise, for instance, from imprecise actuators via probabilities in the transition function. However, in data-driven applications, deriving precise probabilities from (limited) data introduces statistical errors that may lead to unexpected or undesirable outcomes. Uncertain MDPs (uMDPs) do not require precise probabilities but instead use so-called uncertainty sets in the transitions, accounting for such limited data. Tools from the formal verification community efficiently compute robust policies that provably adhere to formal specifications, like safety constraints, under the worst-case instance in the uncertainty set. We continuously learn the transition probabilities of an MDP in a robust anytime-learning approach that combines a dedicated Bayesian inference scheme with the computation of robust policies. In particular, our method (1) approximates p

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keysuilen2022robust

Related papers

• Robust Batch Policy Learning In Markov Decision Processes (2020)0.00
• Twice Regularized Markov Decision Processes: The Equivalence Between Robustness And Regularization (2023)0.00
• Policy Learning For Robust Markov Decision Process With A Mismatched Generative Model (2022)0.00
• Twice Regularized Mdps And The Equivalence Between Robustness And Regularization (2021)0.00
• A Bayesian Approach To Robust Reinforcement Learning (2019)0.00
• Solving Robust Mdps Through No-regret Dynamics (2023)0.00
• Online MDP With Transition Prototypes: A Robust Adaptive Approach (2024)0.00
• Bayesian Learning Of The Optimal Action-value Function In A Markov Decision Process (2025)0.00