Approximating Euclidean By Imprecise Markov Decision Processes

Abstract

Euclidean Markov decision processes are a powerful tool for modeling control problems under uncertainty over continuous domains. Finite state imprecise, Markov decision processes can be used to approximate the behavior of these infinite models. In this paper we address two questions: first, we investigate what kind of approximation guarantees are obtained when the Euclidean process is approximated by finite state approximations induced by increasingly fine partitions of the continuous state space. We show that for cost functions over finite time horizons the approximations become arbitrarily precise. Second, we use imprecise Markov decision process approximations as a tool to analyse and validate cost functions and strategies obtained by reinforcement learning. We find that, on the one hand, our new theoretical results validate basic design choices of a previously proposed reinforcement learning approach. On the other hand, the imprecise Markov decision process approximations reveal so

Stats

• citations9
• S2 citations—
• github stars0
• HF likes0
• heat score7.50
• arxiv keyjaeger2020approximating

Related papers

• Robust Anytime Learning Of Markov Decision Processes (2022)0.00
• Continuous-time Reinforcement Learning: Ellipticity Enables Model-free Value Function Approximation (2026)0.00
• Bayesian Learning Of Optimal Policies In Markov Decision Processes With Countably Infinite State-space (2023)0.00
• Square-root Regret Bounds For Continuous-time Episodic Markov Decision Processes (2022)2.26
• Reinforcement Learning With Unbiased Policy Evaluation And Linear Function Approximation (2022)0.00
• A Policy Gradient Approach For Finite Horizon Constrained Markov Decision Processes (2022)3.58
• On Learning History Based Policies For Controlling Markov Decision Processes (2022)0.00
• Reward-free Model-based Reinforcement Learning With Linear Function Approximation (2021)0.00