A Unified Perspective On Deep Equilibrium Finding

Abstract

Extensive-form games provide a versatile framework for modeling interactions of multiple agents subjected to imperfect observations and stochastic events. In recent years, two paradigms, policy space response oracles (PSRO) and counterfactual regret minimization (CFR), showed that extensive-form games may indeed be solved efficiently. Both of them are capable of leveraging deep neural networks to tackle the scalability issues inherent to extensive-form games and we refer to them as deep equilibrium-finding algorithms. Even though PSRO and CFR share some similarities, they are often regarded as distinct and the answer to the question of which is superior to the other remains ambiguous. Instead of answering this question directly, in this work we propose a unified perspective on deep equilibrium finding that generalizes both PSRO and CFR. Our four main contributions include: i) a novel response oracle (RO) which computes Q values as well as reaching probability values and baseline values

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywang2022a

Related papers

• Achieving Correlated Equilibrium By Studying Opponent's Behavior Through Policy-based Deep Reinforcement Learning (2020)0.00
• Simple Uncoupled No-regret Learning Dynamics For Extensive-form Correlated Equilibrium (2021)6.34
• Strategically Robust Multi-agent Reinforcement Learning With Linear Function Approximation (2026)0.00
• Pipeline PSRO: A Scalable Approach For Finding Approximate Nash Equilibria In Large Games (2020)0.00
• Learning Equilibria In Mean-field Games: Introducing Mean-field PSRO (2021)0.00
• Regret Minimization In Population Network Games: Vanishing Heterogeneity And Convergence To Equilibria (2025)3.58
• Oracles & Followers: Stackelberg Equilibria In Deep Multi-agent Reinforcement Learning (2022)0.00
• Multi-agent Training Beyond Zero-sum With Correlated Equilibrium Meta-solvers (2021)0.00