Monte Carlo Q-learning For General Game Playing

Abstract

After the recent groundbreaking results of AlphaGo, we have seen a strong interest in reinforcement learning in game playing. General Game Playing (GGP) provides a good testbed for reinforcement learning. In GGP, a specification of games rules is given. GGP problems can be solved by reinforcement learning. Q-learning is one of the canonical reinforcement learning methods, and has been used by (Banerjee & Stone, IJCAI 2007) in GGP. In this paper we implement Q-learning in GGP for three small-board games (Tic-Tac-Toe, Connect Four, Hex), to allow comparison to Banerjee et al. As expected, Q-learning converges, although much slower than MCTS. Borrowing an idea from MCTS, we enhance Q-learning with Monte Carlo Search, to give QM-learning. This enhancement improves the performance of pure Q-learning. We believe that QM-learning can also be used to improve performance of reinforcement learning further for larger games, something which we will test in future work.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywang2018monte

Related papers

• Performing Deep Recurrent Double Q-learning For Atari Games (2019)10.07
• Reinforcement Learning For Constraint Satisfaction Game Agents (15-puzzle, Minesweeper, 2048, And Sudoku) (2021)0.00
• Combining Off And On-policy Training In Model-based Reinforcement Learning (2021)0.00
• Reinforcement Learning And Video Games (2019)0.00
• An Evolutionary Framework For Connect-4 As Test-bed For Comparison Of Advanced Minimax, Q-learning And MCTS (2024)0.00
• A General Framework For Learning Mean-field Games (2020)0.00
• Reinforcement Learning In Strategy-based And Atari Games: A Review Of Google Deepminds Innovations (2025)0.00
• Can Meta-interpretive Learning Outperform Deep Reinforcement Learning Of Evaluable Game Strategies? (2019)0.00