Markov Decision Processes Under External Temporal Processes

Abstract

Reinforcement Learning Algorithms are predominantly developed for stationary environments, and the limited literature that considers nonstationary environments often involves specific assumptions about changes that can occur in transition probability matrices and reward functions. Considering that real-world applications involve environments that continuously evolve due to various external events, and humans make decisions by discerning patterns in historical events, this study investigates Markov Decision Processes under the influence of an external temporal process. We establish the conditions under which the problem becomes tractable, allowing it to be addressed by considering only a finite history of events, based on the properties of the perturbations introduced by the exogenous process. We propose and theoretically analyze a policy iteration algorithm to tackle this problem, which learns policies contingent upon the current state of the environment, as well as a finite history of

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyayyagari2023markov

Related papers

• Deep Reinforcement Learning Of Marked Temporal Point Processes (2018)0.00
• Act As You Learn: Adaptive Decision-making In Non-stationary Markov Decision Processes (2024)0.00
• Reinforcement Learning In Switching Non-stationary Markov Decision Processes: Algorithms And Convergence Analysis (2025)0.00
• Policy Gradient For Continuing Tasks In Non-stationary Markov Decision Processes (2020)0.00
• Deterministic Limit Of Temporal Difference Reinforcement Learning For Stochastic Games (2018)12.93
• Regular Decision Processes For Grid Worlds (2021)0.00
• Demystifying Reinforcement Learning In Time-varying Systems (2022)0.00
• Sample-efficient Reinforcement Learning In The Presence Of Exogenous Information (2022)0.00