Decision Making In Non-stationary Environments With Policy-augmented Monte Carlo Tree Search

Abstract

Decision-making under uncertainty (DMU) is present in many important problems. An open challenge is DMU in non-stationary environments, where the dynamics of the environment can change over time. Reinforcement Learning (RL), a popular approach for DMU problems, learns a policy by interacting with a model of the environment offline. Unfortunately, if the environment changes the policy can become stale and take sub-optimal actions, and relearning the policy for the updated environment takes time and computational effort. An alternative is online planning approaches such as Monte Carlo Tree Search (MCTS), which perform their computation at decision time. Given the current environment, MCTS plans using high-fidelity models to determine promising action trajectories. These models can be updated as soon as environmental changes are detected to immediately incorporate them into decision making. However, MCTS's convergence can be slow for domains with large state-action spaces. In this paper,

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keypettet2022decision

Related papers

• Decision Making In Non-stationary Environments With Policy-augmented Search (2024)0.00
• Act As You Learn: Adaptive Decision-making In Non-stationary Markov Decision Processes (2024)0.00
• Policy Gradient Algorithms With Monte Carlo Tree Learning For Non-markov Decision Processes (2022)0.00
• Maneuver Decision-making Through Proximal Policy Optimization And Monte Carlo Tree Search (2023)0.00
• Ordinal Monte Carlo Tree Search (2019)0.00
• Bayes Adaptive Monte Carlo Tree Search For Offline Model-based Reinforcement Learning (2024)0.00
• Risk Aware And Multi-objective Decision Making With Distributional Monte Carlo Tree Search (2021)0.00
• Non-stationary Markov Decision Processes, A Worst-case Approach Using Model-based Reinforcement Learning, Extended Version (2019)0.00