Risk Aware And Multi-objective Decision Making With Distributional Monte Carlo Tree Search

Abstract

In many risk-aware and multi-objective reinforcement learning settings, the utility of the user is derived from the single execution of a policy. In these settings, making decisions based on the average future returns is not suitable. For example, in a medical setting a patient may only have one opportunity to treat their illness. When making a decision, just the expected return -- known in reinforcement learning as the value -- cannot account for the potential range of adverse or positive outcomes a decision may have. Our key insight is that we should use the distribution over expected future returns differently to represent the critical information that the agent requires at decision time. In this paper, we propose Distributional Monte Carlo Tree Search, an algorithm that learns a posterior distribution over the utility of the different possible returns attainable from individual policy executions, resulting in good policies for both risk-aware and multi-objective settings. Moreover,

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyhayes2021risk

Related papers

• Distributional Method For Risk Averse Reinforcement Learning (2023)0.00
• Decision Making In Non-stationary Environments With Policy-augmented Monte Carlo Tree Search (2022)0.00
• A Risk-sensitive Approach To Policy Optimization (2022)3.58
• Ordinal Monte Carlo Tree Search (2019)0.00
• Toward Risk-based Optimistic Exploration For Cooperative Multi-agent Reinforcement Learning (2023)0.00
• Pitfall Of Optimism: Distributional Reinforcement Learning By Randomizing Risk Criterion (2023)0.00
• Distributional Reward Estimation For Effective Multi-agent Deep Reinforcement Learning (2022)0.00
• Distributional Reinforcement Learning For Multi-dimensional Reward Functions (2021)0.00