A Deep Policy Inference Q-network For Multi-agent Systems

Abstract

We present DPIQN, a deep policy inference Q-network that targets multi-agent systems composed of controllable agents, collaborators, and opponents that interact with each other. We focus on one challenging issue in such systems---modeling agents with varying strategies---and propose to employ "policy features" learned from raw observations (e.g., raw images) of collaborators and opponents by inferring their policies. DPIQN incorporates the learned policy features as a hidden vector into its own deep Q-network (DQN), such that it is able to predict better Q values for the controllable agents than the state-of-the-art deep reinforcement learning models. We further propose an enhanced version of DPIQN, called deep recurrent policy inference Q-network (DRPIQN), for handling partial observability. Both DPIQN and DRPIQN are trained by an adaptive training procedure, which adjusts the network's attention to learn the policy features and its own Q-values at different phases of the training pro

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyhong2017a

Related papers

• Weighted Double Deep Multiagent Reinforcement Learning In Stochastic Cooperative Environments (2018)0.00
• Training A Resilient Q-network Against Observational Interference (2021)0.00
• Deep Multi-agent Reinforcement Learning With Discrete-continuous Hybrid Action Spaces (2019)12.47
• Resolving Implicit Coordination In Multi-agent Deep Reinforcement Learning With Deep Q-networks & Game Theory (2020)0.00
• Multi-agent Determinantal Q-learning (2020)0.00
• Deep Active Inference For Partially Observable Mdps (2020)9.59
• Modelling The Dynamic Joint Policy Of Teammates With Attention Multi-agent DDPG (2018)5.84
• Factorized Q-learning For Large-scale Multi-agent Systems (2018)11.58