Combining Off And On-policy Training In Model-based Reinforcement Learning

Abstract

The combination of deep learning and Monte Carlo Tree Search (MCTS) has shown to be effective in various domains, such as board and video games. AlphaGo represented a significant step forward in our ability to learn complex board games, and it was rapidly followed by significant advances, such as AlphaGo Zero and AlphaZero. Recently, MuZero demonstrated that it is possible to master both Atari games and board games by directly learning a model of the environment, which is then used with MCTS to decide what move to play in each position. During tree search, the algorithm simulates games by exploring several possible moves and then picks the action that corresponds to the most promising trajectory. When training, limited use is made of these simulated games since none of their trajectories are directly used as training examples. Even if we consider that not all trajectories from simulated games are useful, there are thousands of potentially useful trajectories that are discarded. Using i

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyborges2021combining

Related papers

• Multiple Policy Value Monte Carlo Tree Search (2019)0.00
• Learning Policies From Self-play With Policy Gradients And MCTS Value Estimates (2019)0.00
• Convex Regularization In Monte-carlo Tree Search (2020)0.00
• Combining Tree-search, Generative Models, And Nash Bargaining Concepts In Game-theoretic Reinforcement Learning (2023)0.00
• Policy Gradient Algorithms With Monte Carlo Tree Learning For Non-markov Decision Processes (2022)0.00
• Variance-aware Prior-based Tree Policies For Monte Carlo Tree Search (2026)0.00
• Decision Making In Non-stationary Environments With Policy-augmented Monte Carlo Tree Search (2022)0.00
• Efficient Offline Policy Optimization With A Learned Model (2022)0.00