Targeted Search Control In Alphazero For Effective Policy Improvement

Abstract

AlphaZero is a self-play reinforcement learning algorithm that achieves superhuman play in chess, shogi, and Go via policy iteration. To be an effective policy improvement operator, AlphaZero's search requires accurate value estimates for the states appearing in its search tree. AlphaZero trains upon self-play matches beginning from the initial state of a game and only samples actions over the first few moves, limiting its exploration of states deeper in the game tree. We introduce Go-Exploit, a novel search control strategy for AlphaZero. Go-Exploit samples the start state of its self-play trajectories from an archive of states of interest. Beginning self-play trajectories from varied starting states enables Go-Exploit to more effectively explore the game tree and to learn a value function that generalizes better. Producing shorter self-play trajectories allows Go-Exploit to train upon more independent value targets, improving value training. Finally, the exploration inherent in Go-Ex

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keytrudeau2023targeted

Related papers

• Regret-guided Search Control For Efficient Learning In Alphazero (2026)0.00
• Policy-value Alignment And Robustness In Search-based Multi-agent Learning (2023)0.00
• ELF Opengo: An Analysis And Open Reimplementation Of Alphazero (2019)0.00
• Are Alphazero-like Agents Robust To Adversarial Perturbations? (2022)0.00
• Analysis Of Hyper-parameters For Small Games: Iterations Or Epochs In Self-play? (2020)0.00
• Adaptable Hindsight Experience Replay For Search-based Learning (2025)0.00
• Go-explore: A New Approach For Hard-exploration Problems (2019)0.00
• Modeling Strong And Human-like Gameplay With Kl-regularized Search (2021)0.00