Human-level Competitive Pok\'emon Via Scalable Offline Reinforcement Learning With Transformers
2025 · Jake Grigsby, Yuqi Xie, Justin Sasek, et al.
Abstract
Competitive Pok\'emon Singles (CPS) is a popular strategy game where players learn to exploit their opponent based on imperfect information in battles that can last more than one hundred stochastic turns. AI research in CPS has been led by heuristic tree search and online self-play, but the game may also create a platform to study adaptive policies trained offline on large datasets. We develop a pipeline to reconstruct the first-person perspective of an agent from logs saved from the third-person perspective of a spectator, thereby unlocking a dataset of real human battles spanning more than a decade that grows larger every day. This dataset enables a black-box approach where we train large sequence models to adapt to their opponent based solely on their input trajectory while selecting moves without explicit search of any kind. We study a progression from imitation learning to offline RL and offline fine-tuning on self-play data in the hardcore competitive setting of Pok\'emon's four
Authors
(none)
Tags
Stats
Related papers
- All By Myself: Learning Individualized Competitive Behaviour With A Contrastive Reinforcement Learning Optimization (2023)7.16
- Offline Fictitious Self-play For Competitive Games (2024)0.00
- Learning From Learners: Adapting Reinforcement Learning Agents To Be Competitive In A Card Game (2020)0.00
- Pytupli: A Scalable Infrastructure For Collaborative Offline Reinforcement Learning Projects (2025)0.00
- Mimicking To Dominate: Imitation Learning Strategies For Success In Multiagent Competitive Games (2023)0.00
- Efficient Competitive Self-play Policy Optimization (2020)0.00
- Self-confirming Transformer For Belief-conditioned Adaptation In Offline Multi-agent Reinforcement Learning (2023)0.00
- Persim: Data-efficient Offline Reinforcement Learning With Heterogeneous Agents Via Personalized Simulators (2021)0.00