Impartial Games: A Challenge For Reinforcement Learning

Abstract

AlphaZero-style reinforcement learning (RL) algorithms have achieved superhuman performance in many complex board games such as Chess, Shogi, and Go. However, we showcase that these algorithms encounter significant and fundamental challenges when applied to impartial games, a class where players share game pieces and optimal strategy often relies on abstract mathematical principles. Specifically, we utilise the game of Nim as a concrete and illustrative case study to reveal critical limitations of AlphaZero-style and similar self-play RL algorithms. We introduce a novel conceptual framework distinguishing between champion and expert mastery to evaluate RL agent performance. Our findings reveal that while AlphaZero-style agents can achieve champion-level play on very small Nim boards, their learning progression severely degrades as the board size increases. This difficulty stems not merely from complex data distributions or noisy labels, but from a deeper representational bottleneck: th

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhou2022impartial

Related papers

• Reinforcement Learning In Strategy-based And Atari Games: A Review Of Google Deepminds Innovations (2025)0.00
• Combining Deep Reinforcement Learning And Search For Imperfect-information Games (2020)0.00
• Score Vs. Winrate In Score-based Games: Which Reward For Reinforcement Learning? (2022)7.16
• Aixijs: A Software Demo For General Reinforcement Learning (2017)0.00
• Exploring Parity Challenges In Reinforcement Learning Through Curriculum Learning With Noisy Labels (2023)2.26
• Modeling Strong And Human-like Gameplay With Kl-regularized Search (2021)0.00
• Colosseumrl: A Framework For Multiagent Reinforcement Learning In \(n\)-player Games (2019)0.00
• Supervised And Reinforcement Learning From Observations In Reconnaissance Blind Chess (2022)7.16