Regular Decision Processes For Grid Worlds

Abstract

Markov decision processes are typically used for sequential decision making under uncertainty. For many aspects however, ranging from constrained or safe specifications to various kinds of temporal (non-Markovian) dependencies in task and reward structures, extensions are needed. To that end, in recent years interest has grown into combinations of reinforcement learning and temporal logic, that is, combinations of flexible behavior learning methods with robust verification and guarantees. In this paper we describe an experimental investigation of the recently introduced regular decision processes that support both non-Markovian reward functions as well as transition functions. In particular, we provide a tool chain for regular decision processes, algorithmic extensions relating to online, incremental learning, an empirical evaluation of model-free and model-based solution algorithms, and applications in regular, but non-Markovian, grid worlds.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylenaers2021regular

Related papers

• Omega-regular Decision Processes (2023)0.00
• Markov Decision Processes Under External Temporal Processes (2023)0.00
• Extrapolation In Gridworld Markov-decision Processes (2020)0.00
• Efficient PAC Reinforcement Learning In Regular Decision Processes (2021)2.26
• Temporal Regularization In Markov Decision Process (2018)0.00
• Act As You Learn: Adaptive Decision-making In Non-stationary Markov Decision Processes (2024)0.00
• Programmatic Reinforcement Learning: Navigating Gridworlds (2024)0.00
• Markov Abstractions For PAC Reinforcement Learning In Non-markov Decision Processes (2022)0.00